[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:31 dist-test-slave-vxj2 krb5kdc[24712](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:31 dist-test-slave-vxj2 krb5kdc[24712](info): set up 2 sockets
Aug 09 22:58:31 dist-test-slave-vxj2 krb5kdc[24712](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 22:58:32.623441 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.003s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:58:32 dist-test-slave-vxj2 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:58:32.817582 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:40119
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:40119
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:33.147560 24726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:33.147850 24726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:33.148056 24726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:33.155378 24726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:33.155494 24726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:33.155617 24726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:33.155717 24726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:33.155813 24726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:33.165913 24726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:40119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:40119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:33.167344 24726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:33.168689 24726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:33.186838 24732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:33.186903 24731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:33.186815 24734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:33.187860 24726 server_base.cc:1034] running on GCE node
I20260809 22:58:33.188722 24726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:33.190302 24726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:33.191555 24726 hybrid_clock.cc:648] HybridClock initialized: now 1786316313191527 us; error 57 us; skew 500 ppm
I20260809 22:58:33.191931 24726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:33 dist-test-slave-vxj2 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:33.195430 24726 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:58:33.197003 24726 webserver.cc:460] Webserver started at http://127.24.31.254:40055/ using document root <none> and password file <none>
I20260809 22:58:33.197793 24726 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:33.197943 24726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:33.198251 24726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:33.201547 24726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "edad5239e173483d991c5a1e24ddc6f8"
format_stamp: "Formatted at 2026-08-09 22:58:33 on dist-test-slave-vxj2"
server_key: "91d802afe68e30d891471af77c4478d6"
server_key_iv: "90a4458b5c342b470e3a4ee70910ff88"
server_key_version: "encryptionkey@0"
I20260809 22:58:33.202353 24726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "edad5239e173483d991c5a1e24ddc6f8"
format_stamp: "Formatted at 2026-08-09 22:58:33 on dist-test-slave-vxj2"
server_key: "91d802afe68e30d891471af77c4478d6"
server_key_iv: "90a4458b5c342b470e3a4ee70910ff88"
server_key_version: "encryptionkey@0"
I20260809 22:58:33.207710 24726 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:58:33.211335 24743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:33.212679 24726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:58:33.212896 24726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "edad5239e173483d991c5a1e24ddc6f8"
format_stamp: "Formatted at 2026-08-09 22:58:33 on dist-test-slave-vxj2"
server_key: "91d802afe68e30d891471af77c4478d6"
server_key_iv: "90a4458b5c342b470e3a4ee70910ff88"
server_key_version: "encryptionkey@0"
I20260809 22:58:33.213133 24726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:33.232237 24726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:33.237218 24726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:33.237533 24726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:33.256726 24726 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:40119
I20260809 22:58:33.256749 24796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:40119 every 8 connection(s)
I20260809 22:58:33.258180 24726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:33.261919 24797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:33.266105 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 24726
I20260809 22:58:33.266465 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:58:33.268929 24703 external_mini_cluster.cc:1428] Setting key bbf22885cca41af2bb6d30dd566e52fc
I20260809 22:58:33.274914 24797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8: Bootstrap starting.
I20260809 22:58:33.277997 24797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:33.279006 24797 log.cc:824] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:33.281770 24797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8: No bootstrap required, opened a new log
I20260809 22:58:33.291265 24797 raft_consensus.cc:348] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad5239e173483d991c5a1e24ddc6f8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40119 } }
I20260809 22:58:33.291730 24797 raft_consensus.cc:374] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:33.291853 24797 raft_consensus.cc:729] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edad5239e173483d991c5a1e24ddc6f8, State: Initialized, Role: FOLLOWER
I20260809 22:58:33.292528 24797 consensus_queue.cc:260] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad5239e173483d991c5a1e24ddc6f8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40119 } }
I20260809 22:58:33.292796 24797 raft_consensus.cc:388] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:33.292937 24797 raft_consensus.cc:482] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:33.293179 24797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:33.295650 24797 raft_consensus.cc:504] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad5239e173483d991c5a1e24ddc6f8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40119 } }
I20260809 22:58:33.296242 24797 leader_election.cc:302] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edad5239e173483d991c5a1e24ddc6f8; no voters:
I20260809 22:58:33.296876 24797 leader_election.cc:288] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:33.297124 24802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:33.298344 24802 raft_consensus.cc:686] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [term 1 LEADER]: Becoming Leader. State: Replica: edad5239e173483d991c5a1e24ddc6f8, State: Running, Role: LEADER
I20260809 22:58:33.299008 24802 consensus_queue.cc:237] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad5239e173483d991c5a1e24ddc6f8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40119 } }
I20260809 22:58:33.299201 24797 sys_catalog.cc:564] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:33.302326 24804 sys_catalog.cc:455] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edad5239e173483d991c5a1e24ddc6f8. Latest consensus state: current_term: 1 leader_uuid: "edad5239e173483d991c5a1e24ddc6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad5239e173483d991c5a1e24ddc6f8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40119 } } }
I20260809 22:58:33.302908 24804 sys_catalog.cc:458] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [sys.catalog]: This master's current role is: LEADER
Aug 09 22:58:33 dist-test-slave-vxj2 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:33.302418 24803 sys_catalog.cc:455] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edad5239e173483d991c5a1e24ddc6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edad5239e173483d991c5a1e24ddc6f8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40119 } } }
I20260809 22:58:33.303575 24803 sys_catalog.cc:458] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:33.312198 24811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:33.319032 24799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:33.273933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39526 (local address 127.24.31.254:40119)
0809 22:58:33.274966 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:33.274983 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:33.276497 (+ 1514us) server_negotiation.cc:407] Connection header received
0809 22:58:33.286785 (+ 10288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:33.286810 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:33.287282 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:33.287715 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:33.290935 (+ 3220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:33.292978 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:33.294930 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:33.296022 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:33.296205 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:33.305278 (+ 9073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:33.305297 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:33.305321 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:33.305391 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:33.311263 (+ 5872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:33.312443 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:33.312453 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:33.312461 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:33.312706 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:33.313437 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:33.313446 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:33.313451 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:33.314461 (+ 1010us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:33.314894 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:33.316374 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0809 22:58:33.316843 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":766,"thread_start_us":521,"threads_started":1}
I20260809 22:58:33.322913 24811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:33.332360 24811 catalog_manager.cc:1347] Generated new cluster ID: 43caa367ae0f4f97afaaba4fbbb82033
I20260809 22:58:33.332480 24811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:33.350606 24811 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:33.351904 24811 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:33.361095 24811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edad5239e173483d991c5a1e24ddc6f8: Generated new TSK 0
I20260809 22:58:33.362021 24811 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:58:33.428848 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40119
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:33.760536 24824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:33.760874 24824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:33.761082 24824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:33.768349 24824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:33.768502 24824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:33.768697 24824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:58:33.778546 24824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40119
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:33.779867 24824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:33.781263 24824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:33.794546 24829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:33.794495 24830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:33.794519 24833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:33.798600 24824 server_base.cc:1034] running on GCE node
I20260809 22:58:33.799304 24824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:33.800419 24824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:33.801671 24824 hybrid_clock.cc:648] HybridClock initialized: now 1786316313801607 us; error 53 us; skew 500 ppm
I20260809 22:58:33.802030 24824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:33 dist-test-slave-vxj2 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:33.805219 24824 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:58:33.806773 24824 webserver.cc:460] Webserver started at http://127.24.31.193:44513/ using document root <none> and password file <none>
I20260809 22:58:33.807541 24824 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:33.807689 24824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:33.808028 24824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:33.811115 24824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "83305c503bab4ad99dd915eb6d52b3cb"
format_stamp: "Formatted at 2026-08-09 22:58:33 on dist-test-slave-vxj2"
server_key: "e7f31f3ba6cfe4da23400801c179a78c"
server_key_iv: "39ba27de6395b30471a59c751eee8f0e"
server_key_version: "encryptionkey@0"
I20260809 22:58:33.811897 24824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "83305c503bab4ad99dd915eb6d52b3cb"
format_stamp: "Formatted at 2026-08-09 22:58:33 on dist-test-slave-vxj2"
server_key: "e7f31f3ba6cfe4da23400801c179a78c"
server_key_iv: "39ba27de6395b30471a59c751eee8f0e"
server_key_version: "encryptionkey@0"
I20260809 22:58:33.817131 24824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 22:58:33.821221 24840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:33.822626 24824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:58:33.822891 24824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "83305c503bab4ad99dd915eb6d52b3cb"
format_stamp: "Formatted at 2026-08-09 22:58:33 on dist-test-slave-vxj2"
server_key: "e7f31f3ba6cfe4da23400801c179a78c"
server_key_iv: "39ba27de6395b30471a59c751eee8f0e"
server_key_version: "encryptionkey@0"
I20260809 22:58:33.823130 24824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:33.872859 24824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:33.877350 24824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:33.877662 24824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:33.878666 24824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:58:33.880131 24824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:33.880241 24824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:33.880414 24824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:33.880487 24824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:33.915759 24824 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:37913
I20260809 22:58:33.915813 24961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:37913 every 8 connection(s)
I20260809 22:58:33.917138 24824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:58:33.920359 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 24824
I20260809 22:58:33.920611 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:58:33.921492 24703 external_mini_cluster.cc:1428] Setting key cdd935118ce5cef0096a222beb538da6
Aug 09 22:58:33 dist-test-slave-vxj2 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:33.939288 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:33.919853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:35769 (local address 127.24.31.254:40119)
0809 22:58:33.920369 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:33.920373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:33.921561 (+ 1188us) server_negotiation.cc:407] Connection header received
0809 22:58:33.923769 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:33.923774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:33.923859 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:33.923988 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:33.925554 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:33.926312 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:33.927963 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:33.928392 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:33.928551 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:33.931977 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:33.931986 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:33.931993 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:33.932039 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:33.935647 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:33.936603 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:33.936610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:33.936614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:33.936791 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:33.937249 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:33.937258 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:33.937263 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:33.937688 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:33.937897 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:33.938561 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 22:58:33.938834 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":290,"threads_started":1}
I20260809 22:58:33.939952 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:33.920542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:40119 (local address 127.24.31.193:35769)
0809 22:58:33.921399 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:33.921446 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:33.923497 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:33.924229 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:33.924245 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:33.924746 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:33.925386 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:33.925405 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:33.926482 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:33.926487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:33.927821 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:33.927833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:33.928626 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:33.928631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:33.928833 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:33.929577 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:33.929604 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:33.931753 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:33.935872 (+ 4119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:33.935879 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:33.935891 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:33.936439 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:33.936907 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:33.936910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:33.936912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:33.937097 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:33.937833 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:33.937839 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:33.938161 (+ 322us) client_negotiation.cc:769] Sending connection context
0809 22:58:33.938389 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 22:58:33.938815 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":329,"threads_started":1}
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
I20260809 22:58:33.941660 24962 heartbeater.cc:347] Connected to a master server at 127.24.31.254:40119
I20260809 22:58:33.941988 24962 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:33.942804 24962 heartbeater.cc:511] Master 127.24.31.254:40119 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
I20260809 22:58:33.945293 24761 ts_manager.cc:194] Registered new tserver with Master: 83305c503bab4ad99dd915eb6d52b3cb (127.24.31.193:37913)
I20260809 22:58:33.947541 24761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:35769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:58:33.968583 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40119
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:34.307283 24968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:34.307585 24968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:34.307787 24968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:34.315755 24968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:34.315943 24968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:34.316184 24968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:58:34.327703 24968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40119
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:34.329111 24968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:34.330586 24968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:34.343914 24973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:34.343954 24976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:34.344022 24974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:34.344931 24968 server_base.cc:1034] running on GCE node
I20260809 22:58:34.345619 24968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:34.346643 24968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:34.347892 24968 hybrid_clock.cc:648] HybridClock initialized: now 1786316314347832 us; error 44 us; skew 500 ppm
I20260809 22:58:34.348265 24968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:34 dist-test-slave-vxj2 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316314, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:34.351668 24968 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:58:34.353263 24968 webserver.cc:460] Webserver started at http://127.24.31.194:40771/ using document root <none> and password file <none>
I20260809 22:58:34.353979 24968 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:34.354135 24968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:34.354496 24968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:34.357640 24968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "78a6ed09ea8048b092ac851bc757fd5b"
format_stamp: "Formatted at 2026-08-09 22:58:34 on dist-test-slave-vxj2"
server_key: "91f8a7940b8ecac8c976f2b76e27e6dd"
server_key_iv: "0e12ee1a3508ca764259b3b4f3e597ec"
server_key_version: "encryptionkey@0"
I20260809 22:58:34.358481 24968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "78a6ed09ea8048b092ac851bc757fd5b"
format_stamp: "Formatted at 2026-08-09 22:58:34 on dist-test-slave-vxj2"
server_key: "91f8a7940b8ecac8c976f2b76e27e6dd"
server_key_iv: "0e12ee1a3508ca764259b3b4f3e597ec"
server_key_version: "encryptionkey@0"
I20260809 22:58:34.363970 24968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:58:34.367703 24985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:34.369055 24968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:58:34.369266 24968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "78a6ed09ea8048b092ac851bc757fd5b"
format_stamp: "Formatted at 2026-08-09 22:58:34 on dist-test-slave-vxj2"
server_key: "91f8a7940b8ecac8c976f2b76e27e6dd"
server_key_iv: "0e12ee1a3508ca764259b3b4f3e597ec"
server_key_version: "encryptionkey@0"
I20260809 22:58:34.369470 24968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:34.392019 24968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:34.396263 24968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:34.396592 24968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:34.397754 24968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:58:34.399185 24968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:34.399291 24968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:34.399488 24968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:34.399549 24968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:34.436642 24968 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:43319
I20260809 22:58:34.436687 25102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:43319 every 8 connection(s)
I20260809 22:58:34.438130 24968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:58:34.448555 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 24968
I20260809 22:58:34.448786 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:58:34.449294 24703 external_mini_cluster.cc:1428] Setting key bbd28dbe21a4e0e2e35cd89d440dccf7
Aug 09 22:58:34 dist-test-slave-vxj2 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316314, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:34.460215 25106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:34.440901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:48375 (local address 127.24.31.254:40119)
0809 22:58:34.441391 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:34.441395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:34.442391 (+ 996us) server_negotiation.cc:407] Connection header received
0809 22:58:34.444309 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:34.444315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:34.444405 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:34.444547 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:34.446148 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:34.446894 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:34.448662 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:34.449288 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:34.449447 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:34.453572 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:34.453578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:34.453581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:34.453616 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:34.456514 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:34.457571 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:34.457577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:34.457588 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:34.457752 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:34.458236 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:34.458248 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:34.458253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:34.458545 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:34.458732 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:34.459520 (+ 788us) server_negotiation.cc:299] Negotiation successful
0809 22:58:34.459775 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":291,"threads_started":1}
I20260809 22:58:34.461297 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:34.441506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:40119 (local address 127.24.31.194:48375)
0809 22:58:34.442232 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:34.442269 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:34.444031 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:34.444819 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:34.444827 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:34.445410 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:34.445962 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:34.445980 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:34.447065 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:34.447069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:34.448437 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:34.448468 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:34.449565 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:34.449573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:34.449856 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:34.450589 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:34.450617 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:34.453314 (+ 2697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:34.456843 (+ 3529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:34.456853 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:34.456870 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:34.457429 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:34.457880 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:34.457887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:34.457890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:34.458098 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:34.458689 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:34.458697 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:34.459075 (+ 378us) client_negotiation.cc:769] Sending connection context
0809 22:58:34.459403 (+ 328us) client_negotiation.cc:240] Negotiation successful
0809 22:58:34.459966 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":296,"threads_started":1}
I20260809 22:58:34.462927 25103 heartbeater.cc:347] Connected to a master server at 127.24.31.254:40119
I20260809 22:58:34.463306 25103 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:34.464269 25103 heartbeater.cc:511] Master 127.24.31.254:40119 requested a full tablet report, sending...
I20260809 22:58:34.466109 24761 ts_manager.cc:194] Registered new tserver with Master: 78a6ed09ea8048b092ac851bc757fd5b (127.24.31.194:43319)
I20260809 22:58:34.467419 24761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:48375
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:58:34.492830 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40119
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:34.822862 25110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:34.823166 25110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:34.823269 25110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:34.830964 25110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:34.831077 25110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:34.831195 25110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:58:34.841131 25110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40119
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:34.842355 25110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:34.843592 25110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:34.856729 25116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:34.856909 25117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:34.856874 25120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:34.857949 25110 server_base.cc:1034] running on GCE node
I20260809 22:58:34.858527 25110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:34.859437 25110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:34.860635 25110 hybrid_clock.cc:648] HybridClock initialized: now 1786316314860569 us; error 57 us; skew 500 ppm
I20260809 22:58:34.860947 25110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:34 dist-test-slave-vxj2 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316314, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:34.864188 25110 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:58:34.865805 25110 webserver.cc:460] Webserver started at http://127.24.31.195:35699/ using document root <none> and password file <none>
I20260809 22:58:34.866466 25110 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:34.866608 25110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:34.867012 25110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:34.869911 25110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "73dae3e64a8949d5b4ba6f6f91cd55fe"
format_stamp: "Formatted at 2026-08-09 22:58:34 on dist-test-slave-vxj2"
server_key: "9398b6fb1f6bdae6622170798c9a79c6"
server_key_iv: "966f45a347eccf668afe1e922a934977"
server_key_version: "encryptionkey@0"
I20260809 22:58:34.870719 25110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "73dae3e64a8949d5b4ba6f6f91cd55fe"
format_stamp: "Formatted at 2026-08-09 22:58:34 on dist-test-slave-vxj2"
server_key: "9398b6fb1f6bdae6622170798c9a79c6"
server_key_iv: "966f45a347eccf668afe1e922a934977"
server_key_version: "encryptionkey@0"
I20260809 22:58:34.876183 25110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:58:34.879812 25127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:34.881266 25110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:34.881443 25110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "73dae3e64a8949d5b4ba6f6f91cd55fe"
format_stamp: "Formatted at 2026-08-09 22:58:34 on dist-test-slave-vxj2"
server_key: "9398b6fb1f6bdae6622170798c9a79c6"
server_key_iv: "966f45a347eccf668afe1e922a934977"
server_key_version: "encryptionkey@0"
I20260809 22:58:34.881618 25110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:34.903505 25110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:34.907763 25110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:34.908087 25110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:34.909097 25110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:58:34.910481 25110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:34.910655 25110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 22:58:34.910827 25110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:34.910904 25110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:34.947916 25110 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:42831
I20260809 22:58:34.947976 25258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:42831 every 8 connection(s)
I20260809 22:58:34.949273 25110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:58:34.951267 24962 heartbeater.cc:503] Master 127.24.31.254:40119 was elected leader, sending a full tablet report...
I20260809 22:58:34.951696 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 25110
I20260809 22:58:34.952144 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:58:34.952693 24703 external_mini_cluster.cc:1428] Setting key b9b29cd13541f0cc480b5a53a6b053ec
Aug 09 22:58:34 dist-test-slave-vxj2 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316314, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:34.970115 25106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:34.952159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:58073 (local address 127.24.31.254:40119)
0809 22:58:34.952353 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:34.952359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:34.953622 (+ 1263us) server_negotiation.cc:407] Connection header received
0809 22:58:34.955847 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:34.955851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:34.956001 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:34.956110 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:34.957807 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:34.958576 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:34.960081 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:34.960485 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:34.960589 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:34.964106 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:34.964112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:34.964116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:34.964150 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:34.966824 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:34.967661 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:34.967671 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:34.967673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:34.967799 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:34.968435 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:34.968438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:34.968440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:34.968804 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:34.968982 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:34.969687 (+ 705us) server_negotiation.cc:299] Negotiation successful
0809 22:58:34.969888 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:58:34.971012 25263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:34.952564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:40119 (local address 127.24.31.195:58073)
0809 22:58:34.953429 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:34.953490 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:34.955498 (+ 2008us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:34.956401 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:34.956413 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:34.956966 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:34.957613 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:34.957631 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:34.958700 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:34.958704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:34.959903 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:34.959915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:34.960675 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:34.960679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:34.960881 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:34.961594 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:34.961615 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:34.963905 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:34.966960 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:34.966969 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:34.966984 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:34.967538 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:34.967933 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:34.967936 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:34.967939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:34.968169 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:34.968960 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:34.968966 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:34.969344 (+ 378us) client_negotiation.cc:769] Sending connection context
0809 22:58:34.969571 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 22:58:34.969936 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":349,"threads_started":1}
I20260809 22:58:34.972337 25260 heartbeater.cc:347] Connected to a master server at 127.24.31.254:40119
I20260809 22:58:34.972654 25260 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:34.973407 25260 heartbeater.cc:511] Master 127.24.31.254:40119 requested a full tablet report, sending...
I20260809 22:58:34.974759 24761 ts_manager.cc:194] Registered new tserver with Master: 73dae3e64a8949d5b4ba6f6f91cd55fe (127.24.31.195:42831)
I20260809 22:58:34.975576 24761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:58073
I20260809 22:58:34.981029 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:58:35 dist-test-slave-vxj2 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316315, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:58:35 dist-test-slave-vxj2 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316315, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:58:35.027860 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:35.012900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.24.31.193:37913)
0809 22:58:35.013662 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:35.013670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:35.013697 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:58:35.013854 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:35.013862 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:35.014093 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:35.014228 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:35.015650 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.016647 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.018032 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.018595 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.018694 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:35.022502 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:35.022513 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:35.022527 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:35.022565 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:35.025308 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.025956 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.025963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.025965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.026088 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.026403 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.026406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.026408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.026743 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:35.027002 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:35.027254 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 22:58:35.027493 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":433,"threads_started":1}
W20260809 22:58:35.029105 24870 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40996
Aug 09 22:58:35 dist-test-slave-vxj2 krb5kdc[24712](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:58:35 dist-test-slave-vxj2 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:58:35.076041 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:35.062433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.24.31.193:37913)
0809 22:58:35.062701 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:35.062706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:35.062838 (+ 132us) server_negotiation.cc:407] Connection header received
0809 22:58:35.062967 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:35.062971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:35.063048 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:35.063137 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:35.064539 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.065321 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.066554 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.067009 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.067110 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:35.070983 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:35.070988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:35.070991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:35.071025 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:35.073813 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.074418 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.074422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.074424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.074552 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.074875 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.074879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.074881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.075138 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:35.075330 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:35.075524 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 22:58:35.075686 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 22:58:35.077608 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 24824
I20260809 22:58:35.090499 25268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:35.078712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.24.31.193:37913)
0809 22:58:35.078914 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:35.078919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:35.079118 (+ 199us) server_negotiation.cc:407] Connection header received
0809 22:58:35.079297 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:35.079302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:35.079356 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:35.079456 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:35.080601 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.081349 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.082600 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.083008 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.083158 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:35.085438 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:35.085442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:35.085446 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:35.085475 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:35.087921 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.088565 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.088571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.088575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.088710 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.089065 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.089073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.089077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.089393 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:35.089636 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:35.089933 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 22:58:35.090147 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:58:35.091243 24957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:35.176695 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 24824
I20260809 22:58:35.195463 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 24968
Aug 09 22:58:35 dist-test-slave-vxj2 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 22:58:35.211800 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:35.196354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.24.31.194:43319)
0809 22:58:35.196926 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:35.196937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:35.196973 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:58:35.197035 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:35.197041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:35.197388 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:35.197527 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:35.198702 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.199746 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.201114 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.201718 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.201831 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:35.205942 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:35.205953 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:35.205966 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:35.206000 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:35.208756 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.209539 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.209547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.209549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.209673 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.210081 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.210087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.210090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.210497 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:35.210817 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:35.211135 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 22:58:35.211424 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":299,"threads_started":1}
I20260809 22:58:35.213073 25097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:35.295899 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 24968
I20260809 22:58:35.314651 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 25110
Aug 09 22:58:35 dist-test-slave-vxj2 krb5kdc[24712](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 22:58:35.332612 25278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:35.315803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59918 (local address 127.24.31.195:42831)
0809 22:58:35.316548 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:35.316558 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:35.316597 (+ 39us) server_negotiation.cc:407] Connection header received
0809 22:58:35.316659 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:35.316666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:35.316954 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:35.317082 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:35.318457 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.319840 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.321939 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:35.322823 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:35.322926 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:35.327248 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:35.327263 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:35.327278 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:35.327318 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:35.330245 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.330874 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.330881 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.330884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.331005 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:35.331320 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:35.331323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:35.331325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:35.331647 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:35.331884 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:35.332128 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 22:58:35.332326 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":375,"threads_started":1}
I20260809 22:58:35.333959 25254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:35.420122 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 25110
I20260809 22:58:35.439973 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 24726
I20260809 22:58:35.441401 24792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:35.546617 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 24726
2026-08-09T22:58:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4135 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:35 dist-test-slave-vxj2 krb5kdc[25283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:35 dist-test-slave-vxj2 krb5kdc[25283](info): set up 2 sockets
Aug 09 22:58:35 dist-test-slave-vxj2 krb5kdc[25283](info): commencing operation
krb5kdc: starting...
W20260809 22:58:36.701538 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:58:36 dist-test-slave-vxj2 krb5kdc[25283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:58:36.890861 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:41347
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:41347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:37.211380 25298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:37.211725 25298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:37.211898 25298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:37.219050 25298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:37.219197 25298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:37.219313 25298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:37.219411 25298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:37.219501 25298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:37.229417 25298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:41347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:41347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:37.230768 25298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:37.232051 25298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:37.244020 25303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:37.243994 25304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:37.244009 25306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:37.245261 25298 server_base.cc:1034] running on GCE node
I20260809 22:58:37.246115 25298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:37.247493 25298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:37.248723 25298 hybrid_clock.cc:648] HybridClock initialized: now 1786316317248683 us; error 53 us; skew 500 ppm
I20260809 22:58:37.249063 25298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:37 dist-test-slave-vxj2 krb5kdc[25283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316317, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:37.257689 25298 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:58:37.259625 25298 webserver.cc:460] Webserver started at http://127.24.31.254:34307/ using document root <none> and password file <none>
I20260809 22:58:37.260286 25298 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:37.260391 25298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:37.260664 25298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:37.263656 25298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "c76b984e7f144bdaa15335918f659814"
format_stamp: "Formatted at 2026-08-09 22:58:37 on dist-test-slave-vxj2"
server_key: "bce0cdff748ff619196f83fa2dad0557"
server_key_iv: "b46ef400b8b5f719f9cd8d1e34fd7019"
server_key_version: "encryptionkey@0"
I20260809 22:58:37.264359 25298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "c76b984e7f144bdaa15335918f659814"
format_stamp: "Formatted at 2026-08-09 22:58:37 on dist-test-slave-vxj2"
server_key: "bce0cdff748ff619196f83fa2dad0557"
server_key_iv: "b46ef400b8b5f719f9cd8d1e34fd7019"
server_key_version: "encryptionkey@0"
I20260809 22:58:37.269564 25298 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:58:37.272884 25313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:37.274205 25298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:37.274346 25298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "c76b984e7f144bdaa15335918f659814"
format_stamp: "Formatted at 2026-08-09 22:58:37 on dist-test-slave-vxj2"
server_key: "bce0cdff748ff619196f83fa2dad0557"
server_key_iv: "b46ef400b8b5f719f9cd8d1e34fd7019"
server_key_version: "encryptionkey@0"
I20260809 22:58:37.274529 25298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:37.293478 25298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:37.297695 25298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:37.297919 25298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:37.316632 25298 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:41347
I20260809 22:58:37.316653 25370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:41347 every 8 connection(s)
I20260809 22:58:37.318019 25298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:37.321852 25371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:37.326717 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 25298
I20260809 22:58:37.326970 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:58:37.327800 24703 external_mini_cluster.cc:1428] Setting key 96cae7d55ea5dc333345a9d007872f7d
I20260809 22:58:37.333658 25371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814: Bootstrap starting.
I20260809 22:58:37.337276 25371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:37.338531 25371 log.cc:824] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:37.341593 25371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814: No bootstrap required, opened a new log
Aug 09 22:58:37 dist-test-slave-vxj2 krb5kdc[25283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:37.351910 25371 raft_consensus.cc:348] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b984e7f144bdaa15335918f659814" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 41347 } }
I20260809 22:58:37.352376 25371 raft_consensus.cc:374] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:37.352492 25371 raft_consensus.cc:729] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c76b984e7f144bdaa15335918f659814, State: Initialized, Role: FOLLOWER
I20260809 22:58:37.352708 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:37.330489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.24.31.254:41347)
0809 22:58:37.331364 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:37.331376 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:37.331416 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:58:37.333087 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:37.333107 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:37.333550 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:37.333901 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:37.335870 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:37.337140 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:37.338916 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:37.339630 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:37.339777 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:37.343929 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:37.343941 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:37.343967 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:37.344020 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:37.348141 (+ 4121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:37.348831 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:37.348839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:37.348847 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:37.349062 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:37.349480 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:37.349485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:37.349489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:37.350220 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:37.350512 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:37.350929 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 22:58:37.351372 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":378,"threads_started":1}
I20260809 22:58:37.353200 25371 consensus_queue.cc:260] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b984e7f144bdaa15335918f659814" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 41347 } }
I20260809 22:58:37.353484 25371 raft_consensus.cc:388] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:37.353657 25371 raft_consensus.cc:482] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:37.353832 25371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:37.356873 25371 raft_consensus.cc:504] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b984e7f144bdaa15335918f659814" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 41347 } }
I20260809 22:58:37.357434 25371 leader_election.cc:302] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c76b984e7f144bdaa15335918f659814; no voters:
I20260809 22:58:37.358064 25371 leader_election.cc:288] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:37.358228 25376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:37.358991 25376 raft_consensus.cc:686] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [term 1 LEADER]: Becoming Leader. State: Replica: c76b984e7f144bdaa15335918f659814, State: Running, Role: LEADER
I20260809 22:58:37.359555 25376 consensus_queue.cc:237] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b984e7f144bdaa15335918f659814" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 41347 } }
I20260809 22:58:37.360114 25371 sys_catalog.cc:564] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:37.362977 25378 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c76b984e7f144bdaa15335918f659814. Latest consensus state: current_term: 1 leader_uuid: "c76b984e7f144bdaa15335918f659814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b984e7f144bdaa15335918f659814" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 41347 } } }
I20260809 22:58:37.363070 25377 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c76b984e7f144bdaa15335918f659814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b984e7f144bdaa15335918f659814" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 41347 } } }
I20260809 22:58:37.363448 25378 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:37.363468 25377 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:37.364914 25385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:37.370260 25385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:37.379221 25385 catalog_manager.cc:1347] Generated new cluster ID: 583d62b037f24695937091c491ab5995
I20260809 22:58:37.379340 25385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:37.397145 25385 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:37.398800 25385 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:37.406265 25385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c76b984e7f144bdaa15335918f659814: Generated new TSK 0
I20260809 22:58:37.407063 25385 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:58:37.459117 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:41347
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:37.803184 25398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:37.803453 25398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:37.803617 25398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:37.810896 25398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:37.811038 25398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:37.811239 25398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:58:37.820946 25398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:41347
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:37.822228 25398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:37.823560 25398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:37.836527 25404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:37.836556 25403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:37.836592 25406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:37.837448 25398 server_base.cc:1034] running on GCE node
I20260809 22:58:37.837982 25398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:37.838877 25398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:37.840129 25398 hybrid_clock.cc:648] HybridClock initialized: now 1786316317840081 us; error 49 us; skew 500 ppm
I20260809 22:58:37.840420 25398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:37 dist-test-slave-vxj2 krb5kdc[25283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316317, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:37.843430 25398 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:58:37.844856 25398 webserver.cc:460] Webserver started at http://127.24.31.193:34349/ using document root <none> and password file <none>
I20260809 22:58:37.845551 25398 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:37.845652 25398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:37.845917 25398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:37.848675 25398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "f3c0fc40e8ae4a20a81c1219b17672a9"
format_stamp: "Formatted at 2026-08-09 22:58:37 on dist-test-slave-vxj2"
server_key: "f46d9a384116514057ed3ea5f511bdd3"
server_key_iv: "2cbf4b433c4c7d6bee4224ce9f0eeac4"
server_key_version: "encryptionkey@0"
I20260809 22:58:37.849574 25398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "f3c0fc40e8ae4a20a81c1219b17672a9"
format_stamp: "Formatted at 2026-08-09 22:58:37 on dist-test-slave-vxj2"
server_key: "f46d9a384116514057ed3ea5f511bdd3"
server_key_iv: "2cbf4b433c4c7d6bee4224ce9f0eeac4"
server_key_version: "encryptionkey@0"
I20260809 22:58:37.854768 25398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 22:58:37.858225 25416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:37.859689 25398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:58:37.859823 25398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "f3c0fc40e8ae4a20a81c1219b17672a9"
format_stamp: "Formatted at 2026-08-09 22:58:37 on dist-test-slave-vxj2"
server_key: "f46d9a384116514057ed3ea5f511bdd3"
server_key_iv: "2cbf4b433c4c7d6bee4224ce9f0eeac4"
server_key_version: "encryptionkey@0"
I20260809 22:58:37.859987 25398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:37.878754 25398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:37.882769 25398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:37.883014 25398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:37.883929 25398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:58:37.885321 25398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:37.885396 25398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:37.885483 25398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:37.885525 25398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:37.918903 25398 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:43225
I20260809 22:58:37.918955 25541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:43225 every 8 connection(s)
I20260809 22:58:37.920228 25398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:58:37.930668 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 25398
I20260809 22:58:37.930891 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:58:37.931730 24703 external_mini_cluster.cc:1428] Setting key de47b0126b3c7b6a7dc7148fdf3b97f9
Aug 09 22:58:37 dist-test-slave-vxj2 krb5kdc[25283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316317, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:37.940929 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:37.922974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:44441 (local address 127.24.31.254:41347)
0809 22:58:37.923423 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:37.923428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:37.924374 (+ 946us) server_negotiation.cc:407] Connection header received
0809 22:58:37.926302 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:37.926307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:37.926399 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:37.926549 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:37.928026 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:37.928751 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:37.930480 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:37.931011 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:37.931124 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:37.935221 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:37.935226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:37.935230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:37.935261 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:37.937907 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:37.938629 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:37.938642 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:37.938646 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:37.938768 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:37.939206 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:37.939211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:37.939215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:37.939475 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:37.939630 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:37.940318 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 22:58:37.940548 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":225,"threads_started":1}
I20260809 22:58:37.941885 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:37.923546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:41347 (local address 127.24.31.193:44441)
0809 22:58:37.924217 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:37.924261 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:37.926034 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:37.926784 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:37.926791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:37.927303 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:37.927842 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:37.927859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:37.928923 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:37.928927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:37.930295 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:37.930313 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:37.931261 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:37.931269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:37.931580 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:37.932428 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:37.932458 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:37.934994 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:37.938051 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:37.938060 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:37.938076 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:37.938499 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:37.938880 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:37.938885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:37.938889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:37.939086 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:37.939605 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:37.939612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:37.939951 (+ 339us) client_negotiation.cc:769] Sending connection context
0809 22:58:37.940213 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 22:58:37.940740 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":267,"threads_started":1}
I20260809 22:58:37.943481 25543 heartbeater.cc:347] Connected to a master server at 127.24.31.254:41347
I20260809 22:58:37.943781 25543 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:37.944504 25543 heartbeater.cc:511] Master 127.24.31.254:41347 requested a full tablet report, sending...
I20260809 22:58:37.946543 25333 ts_manager.cc:194] Registered new tserver with Master: f3c0fc40e8ae4a20a81c1219b17672a9 (127.24.31.193:43225)
I20260809 22:58:37.947961 25333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:44441
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:58:37.975749 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:41347
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:38.298429 25549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:38.298691 25549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:38.298887 25549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:38.306137 25549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:38.306238 25549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:38.306445 25549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:58:38.316299 25549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:41347
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:38.317591 25549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:38.318924 25549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:38.333550 25555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:38.334277 25554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:38.334553 25557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:38.335759 25549 server_base.cc:1034] running on GCE node
I20260809 22:58:38.336328 25549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:38.337448 25549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:38.338650 25549 hybrid_clock.cc:648] HybridClock initialized: now 1786316318338589 us; error 54 us; skew 500 ppm
I20260809 22:58:38.338960 25549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:38 dist-test-slave-vxj2 krb5kdc[25283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:38.342099 25549 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:58:38.343580 25549 webserver.cc:460] Webserver started at http://127.24.31.194:34951/ using document root <none> and password file <none>
I20260809 22:58:38.344259 25549 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:38.344401 25549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:38.344712 25549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:38.347734 25549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "66957ead7ac94e6e9a377aa45a9c8202"
format_stamp: "Formatted at 2026-08-09 22:58:38 on dist-test-slave-vxj2"
server_key: "4f3aee19a975dd686706f5f7485a8fb4"
server_key_iv: "c3e6b59c3d2db381ca3ba457827b4657"
server_key_version: "encryptionkey@0"
I20260809 22:58:38.348645 25549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "66957ead7ac94e6e9a377aa45a9c8202"
format_stamp: "Formatted at 2026-08-09 22:58:38 on dist-test-slave-vxj2"
server_key: "4f3aee19a975dd686706f5f7485a8fb4"
server_key_iv: "c3e6b59c3d2db381ca3ba457827b4657"
server_key_version: "encryptionkey@0"
I20260809 22:58:38.353905 25549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:58:38.357533 25564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:38.358850 25549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:58:38.359031 25549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "66957ead7ac94e6e9a377aa45a9c8202"
format_stamp: "Formatted at 2026-08-09 22:58:38 on dist-test-slave-vxj2"
server_key: "4f3aee19a975dd686706f5f7485a8fb4"
server_key_iv: "c3e6b59c3d2db381ca3ba457827b4657"
server_key_version: "encryptionkey@0"
I20260809 22:58:38.359277 25549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:38.385560 25549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:38.389639 25549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:38.389950 25549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:38.390928 25549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:58:38.392285 25549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:38.392361 25549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:38.392510 25549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:38.392573 25549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:38.426072 25549 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:42069
I20260809 22:58:38.426121 25691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:42069 every 8 connection(s)
I20260809 22:58:38.427387 25549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:58:38.434566 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 25549
I20260809 22:58:38.434777 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:58:38.435352 24703 external_mini_cluster.cc:1428] Setting key 6510c433835ff7424d2cdfdd6270a59e
Aug 09 22:58:38 dist-test-slave-vxj2 krb5kdc[25283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:38.448658 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.429987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:40559 (local address 127.24.31.254:41347)
0809 22:58:38.430170 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:38.430175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:38.431314 (+ 1139us) server_negotiation.cc:407] Connection header received
0809 22:58:38.433192 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:38.433198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:38.433293 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:38.433410 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:38.435229 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.436061 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.438040 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.438484 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.438578 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:38.442765 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:38.442770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:38.442773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:38.442811 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:38.445722 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:38.446405 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:38.446411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:38.446415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:38.446560 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:38.446986 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:38.446993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:38.446997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:38.447254 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:38.447414 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:38.448087 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 22:58:38.448322 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:58:38.449668 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.430470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:41347 (local address 127.24.31.194:40559)
0809 22:58:38.431147 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:38.431189 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:38.432894 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:38.433640 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:38.433649 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:38.434212 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:38.435044 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:38.435071 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.436252 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.436259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:38.437871 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:38.437893 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.438717 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.438728 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:38.439069 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:38.439794 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:38.439828 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:38.442524 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:38.445863 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:38.445872 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:38.445887 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:38.446287 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:38.446675 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:38.446680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:38.446684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:38.446873 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:38.447389 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:38.447396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:38.447733 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 22:58:38.448004 (+ 271us) client_negotiation.cc:240] Negotiation successful
0809 22:58:38.448500 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":268,"threads_started":1}
I20260809 22:58:38.451220 25692 heartbeater.cc:347] Connected to a master server at 127.24.31.254:41347
I20260809 22:58:38.451529 25692 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:38.452204 25692 heartbeater.cc:511] Master 127.24.31.254:41347 requested a full tablet report, sending...
I20260809 22:58:38.453658 25333 ts_manager.cc:194] Registered new tserver with Master: 66957ead7ac94e6e9a377aa45a9c8202 (127.24.31.194:42069)
I20260809 22:58:38.454463 25333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:40559
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:58:38.479991 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:41347
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:38.800112 25697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:38.800382 25697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:38.800550 25697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:38.807677 25697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:38.807787 25697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:38.808002 25697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:58:38.817597 25697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:41347
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:38.818894 25697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:38.820215 25697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:38.833033 25702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:38.833137 25705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:38.833201 25703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:38.834048 25697 server_base.cc:1034] running on GCE node
I20260809 22:58:38.834616 25697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:38.835494 25697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:38.836719 25697 hybrid_clock.cc:648] HybridClock initialized: now 1786316318836655 us; error 75 us; skew 500 ppm
I20260809 22:58:38.837025 25697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:38 dist-test-slave-vxj2 krb5kdc[25283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:38.840114 25697 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:58:38.841583 25697 webserver.cc:460] Webserver started at http://127.24.31.195:39179/ using document root <none> and password file <none>
I20260809 22:58:38.842190 25697 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:38.842289 25697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:38.842542 25697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:38.845309 25697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "bd0481bf97b947ff87ce8fa724b49bd4"
format_stamp: "Formatted at 2026-08-09 22:58:38 on dist-test-slave-vxj2"
server_key: "45cc28a27ef8aba75218bbfe5f098fe0"
server_key_iv: "b3289ed2c8b2881d89a44da3521d09b1"
server_key_version: "encryptionkey@0"
I20260809 22:58:38.846043 25697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "bd0481bf97b947ff87ce8fa724b49bd4"
format_stamp: "Formatted at 2026-08-09 22:58:38 on dist-test-slave-vxj2"
server_key: "45cc28a27ef8aba75218bbfe5f098fe0"
server_key_iv: "b3289ed2c8b2881d89a44da3521d09b1"
server_key_version: "encryptionkey@0"
I20260809 22:58:38.850817 25697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:58:38.854182 25712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:38.855546 25697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:58:38.855677 25697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "bd0481bf97b947ff87ce8fa724b49bd4"
format_stamp: "Formatted at 2026-08-09 22:58:38 on dist-test-slave-vxj2"
server_key: "45cc28a27ef8aba75218bbfe5f098fe0"
server_key_iv: "b3289ed2c8b2881d89a44da3521d09b1"
server_key_version: "encryptionkey@0"
I20260809 22:58:38.855844 25697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:38.874529 25697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:38.878594 25697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:38.878834 25697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:38.879801 25697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:58:38.881080 25697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:38.881174 25697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:38.881265 25697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:38.881310 25697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:38.914261 25697 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:33425
I20260809 22:58:38.914301 25840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:33425 every 8 connection(s)
I20260809 22:58:38.915585 25697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:58:38.916066 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 25697
I20260809 22:58:38.916290 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:58:38.916859 24703 external_mini_cluster.cc:1428] Setting key 6fe6028854d2818d783291d47523a5ca
Aug 09 22:58:38 dist-test-slave-vxj2 krb5kdc[25283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:38.936125 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.918605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:50315 (local address 127.24.31.254:41347)
0809 22:58:38.919103 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:38.919110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:38.920238 (+ 1128us) server_negotiation.cc:407] Connection header received
0809 22:58:38.922217 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:38.922222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:38.922350 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:38.922488 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:38.923933 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.924682 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.926311 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.926753 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.926891 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:38.930326 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:38.930330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:38.930334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:38.930367 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:38.933592 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:38.934241 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:38.934244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:38.934247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:38.934371 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:38.934735 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:38.934738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:38.934740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:38.935006 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:38.935112 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:38.935722 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 22:58:38.935875 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341}
I20260809 22:58:38.937009 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.919145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:41347 (local address 127.24.31.195:50315)
0809 22:58:38.920081 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:38.920134 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:38.921964 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:38.922713 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:38.922721 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:38.923222 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:38.923752 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:38.923769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.924834 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.924843 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:38.926163 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:38.926176 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.927021 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.927029 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:38.927249 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:38.927921 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:38.927948 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:38.930116 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:38.933715 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:38.933721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:38.933732 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:38.934132 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:38.934464 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:38.934467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:38.934469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:38.934636 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:38.935109 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:38.935114 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:38.935396 (+ 282us) client_negotiation.cc:769] Sending connection context
0809 22:58:38.935624 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 22:58:38.935980 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":379,"threads_started":1}
I20260809 22:58:38.938311 25841 heartbeater.cc:347] Connected to a master server at 127.24.31.254:41347
I20260809 22:58:38.938618 25841 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:38.939337 25841 heartbeater.cc:511] Master 127.24.31.254:41347 requested a full tablet report, sending...
I20260809 22:58:38.940686 25333 ts_manager.cc:194] Registered new tserver with Master: bd0481bf97b947ff87ce8fa724b49bd4 (127.24.31.195:33425)
I20260809 22:58:38.941555 25333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:50315
I20260809 22:58:38.944970 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:58:38.950704 25543 heartbeater.cc:503] Master 127.24.31.254:41347 was elected leader, sending a full tablet report...
I20260809 22:58:38.965497 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.951159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.24.31.254:41347)
0809 22:58:38.951328 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:38.951339 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:38.951960 (+ 621us) server_negotiation.cc:407] Connection header received
0809 22:58:38.952145 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:38.952150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:38.952231 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:38.952333 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:38.953960 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.954695 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.955971 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:38.956458 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:38.956599 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:38.959197 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:38.959203 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:38.959209 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:38.959248 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:38.962027 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:38.962783 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:38.962788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:38.962790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:38.962964 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:38.963404 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:38.963408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:38.963410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:38.963914 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:38.964167 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:38.965027 (+ 860us) server_negotiation.cc:299] Negotiation successful
0809 22:58:38.965254 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:58:38.971764 25333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56816:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:58:38.978593 25333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:58:39.026130 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.991892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.24.31.193:43225)
0809 22:58:39.014701 (+ 22809us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.014716 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.017601 (+ 2885us) server_negotiation.cc:407] Connection header received
0809 22:58:39.017742 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.017751 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.018057 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.018267 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:39.019157 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.020904 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.023716 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.025064 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.025250 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.025529 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.025620 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.025796 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22546,"thread_start_us":22839,"threads_started":1}
I20260809 22:58:39.026183 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.991351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:43225 (local address 127.0.0.1:38844)
0809 22:58:39.017474 (+ 26123us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:39.017493 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:39.017607 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:39.018419 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:39.018431 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:39.018476 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:39.018894 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.018931 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.021071 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.021077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.023557 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.023573 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.025284 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.025289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.025430 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.025472 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:58:39.025562 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 22:58:39.025687 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25991,"mutex_wait_us":109,"thread_start_us":24689,"threads_started":1}
I20260809 22:58:39.026899 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.990504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:42069 (local address 127.0.0.1:55436)
0809 22:58:39.015541 (+ 25037us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:39.015592 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:39.015789 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:39.018985 (+ 3196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:39.018989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:39.019019 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:39.019467 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.019480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.021378 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.021385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.024058 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.024071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.025954 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.025960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.026161 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.026203 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:58:39.026299 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 22:58:39.026430 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24747,"mutex_wait_us":112,"thread_start_us":26386,"threads_started":1}
I20260809 22:58:39.027760 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.990679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.24.31.194:42069)
0809 22:58:39.018310 (+ 27631us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.018319 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.018348 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:58:39.018445 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.018451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.018681 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.018968 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:39.019697 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.021248 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.024225 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.025736 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.025895 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.026081 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.027257 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.027421 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27455,"thread_start_us":27320,"threads_started":1}
I20260809 22:58:39.029143 25854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.991034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:33425 (local address 127.0.0.1:37558)
0809 22:58:39.017216 (+ 26182us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:39.017240 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:39.017382 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:39.019532 (+ 2150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:39.019536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:39.019561 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:39.019967 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.019977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.021519 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.021525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.024055 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.024071 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.028287 (+ 4216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.028294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.028482 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.028522 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 22:58:39.028626 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 22:58:39.028753 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25999,"mutex_wait_us":112,"thread_start_us":26635,"threads_started":1}
I20260809 22:58:39.029199 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:38.991112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.24.31.195:33425)
0809 22:58:39.018997 (+ 27885us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.019006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.019040 (+ 34us) server_negotiation.cc:407] Connection header received
0809 22:58:39.019134 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.019141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.019339 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.019531 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:39.020144 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.021363 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.026176 (+ 4813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.028093 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.028237 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.028456 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.028668 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.028828 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27689,"thread_start_us":27499,"threads_started":1}
I20260809 22:58:39.030742 25469 tablet_service.cc:1467] Processing CreateTablet for tablet f5519c1e91a5437d88c6f91003e33fa6 (DEFAULT_TABLE table=test-table [id=8473fb09fddf441593aa9828bf87fd91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:39.031257 25619 tablet_service.cc:1467] Processing CreateTablet for tablet f5519c1e91a5437d88c6f91003e33fa6 (DEFAULT_TABLE table=test-table [id=8473fb09fddf441593aa9828bf87fd91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:39.032289 25469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5519c1e91a5437d88c6f91003e33fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:39.032505 25619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5519c1e91a5437d88c6f91003e33fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:39.032398 25771 tablet_service.cc:1467] Processing CreateTablet for tablet f5519c1e91a5437d88c6f91003e33fa6 (DEFAULT_TABLE table=test-table [id=8473fb09fddf441593aa9828bf87fd91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:39.033589 25771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5519c1e91a5437d88c6f91003e33fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:39.044593 25862 tablet_bootstrap.cc:492] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4: Bootstrap starting.
I20260809 22:58:39.046873 25863 tablet_bootstrap.cc:492] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202: Bootstrap starting.
I20260809 22:58:39.048427 25864 tablet_bootstrap.cc:492] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9: Bootstrap starting.
I20260809 22:58:39.049170 25862 tablet_bootstrap.cc:654] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:39.050370 25863 tablet_bootstrap.cc:654] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:39.050463 25862 log.cc:824] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:39.051412 25864 tablet_bootstrap.cc:654] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:39.051604 25863 log.cc:824] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:39.052665 25864 log.cc:824] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:39.053766 25862 tablet_bootstrap.cc:492] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4: No bootstrap required, opened a new log
I20260809 22:58:39.054134 25862 ts_tablet_manager.cc:1397] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 22:58:39.054656 25863 tablet_bootstrap.cc:492] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202: No bootstrap required, opened a new log
I20260809 22:58:39.054914 25863 ts_tablet_manager.cc:1397] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 22:58:39.055418 25864 tablet_bootstrap.cc:492] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9: No bootstrap required, opened a new log
I20260809 22:58:39.055668 25864 ts_tablet_manager.cc:1397] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 22:58:39.063448 25864 raft_consensus.cc:348] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } }
I20260809 22:58:39.063939 25864 raft_consensus.cc:374] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:39.064041 25864 raft_consensus.cc:729] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c0fc40e8ae4a20a81c1219b17672a9, State: Initialized, Role: FOLLOWER
I20260809 22:58:39.064730 25864 consensus_queue.cc:260] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } }
I20260809 22:58:39.065882 25862 raft_consensus.cc:348] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } }
I20260809 22:58:39.066298 25862 raft_consensus.cc:374] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:39.066401 25862 raft_consensus.cc:729] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0481bf97b947ff87ce8fa724b49bd4, State: Initialized, Role: FOLLOWER
I20260809 22:58:39.066448 25864 ts_tablet_manager.cc:1428] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9: Time spent starting tablet: real 0.011s user 0.009s sys 0.002s
I20260809 22:58:39.067026 25862 consensus_queue.cc:260] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } }
I20260809 22:58:39.067247 25863 raft_consensus.cc:348] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } }
I20260809 22:58:39.067673 25863 raft_consensus.cc:374] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:39.067776 25863 raft_consensus.cc:729] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66957ead7ac94e6e9a377aa45a9c8202, State: Initialized, Role: FOLLOWER
I20260809 22:58:39.068117 25841 heartbeater.cc:503] Master 127.24.31.254:41347 was elected leader, sending a full tablet report...
I20260809 22:58:39.068446 25863 consensus_queue.cc:260] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } }
I20260809 22:58:39.070124 25692 heartbeater.cc:503] Master 127.24.31.254:41347 was elected leader, sending a full tablet report...
I20260809 22:58:39.070219 25863 ts_tablet_manager.cc:1428] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 22:58:39.072377 25862 ts_tablet_manager.cc:1428] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4: Time spent starting tablet: real 0.018s user 0.012s sys 0.000s
I20260809 22:58:39.109422 25869 raft_consensus.cc:482] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:39.109714 25869 raft_consensus.cc:504] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } }
I20260809 22:58:39.111402 25869 leader_election.cc:288] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66957ead7ac94e6e9a377aa45a9c8202 (127.24.31.194:42069), f3c0fc40e8ae4a20a81c1219b17672a9 (127.24.31.193:43225)
I20260809 22:58:39.117142 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.112116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:39251 (local address 127.24.31.193:43225)
0809 22:58:39.112308 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.112313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.112330 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:58:39.112389 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.112393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.112447 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.112556 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:39.112985 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.113759 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.115326 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.116323 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.116489 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.116580 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.116873 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.116962 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:58:39.117259 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.111656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:43225 (local address 127.24.31.195:39251)
0809 22:58:39.111910 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:39.111935 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:39.112067 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:39.112594 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:39.112600 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:39.112630 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:39.112825 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.112834 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.113894 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.113902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.115154 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.115166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.116496 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.116504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.116690 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.116729 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 22:58:39.116826 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 22:58:39.116964 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 22:58:39.118345 25492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5519c1e91a5437d88c6f91003e33fa6" candidate_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" is_pre_election: true
I20260809 22:58:39.118712 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.111864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:44247 (local address 127.24.31.194:42069)
0809 22:58:39.112145 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.112154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.112810 (+ 656us) server_negotiation.cc:407] Connection header received
0809 22:58:39.113037 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.113041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.113103 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.113243 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:39.113826 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.114517 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.116551 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.117807 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.117992 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.118062 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.118330 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.118455 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260809 22:58:39.118757 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.111683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:42069 (local address 127.24.31.195:44247)
0809 22:58:39.112690 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:39.112705 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:39.112867 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:39.113264 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:39.113268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:39.113294 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:39.113647 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.113656 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.114687 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.114693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.116348 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:39.116360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.117999 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.118005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:39.118168 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.118191 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:58:39.118300 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 22:58:39.118420 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":678,"threads_started":1}
I20260809 22:58:39.118935 25492 raft_consensus.cc:2454] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd0481bf97b947ff87ce8fa724b49bd4 in term 0.
I20260809 22:58:39.119700 25714 leader_election.cc:302] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd0481bf97b947ff87ce8fa724b49bd4, f3c0fc40e8ae4a20a81c1219b17672a9; no voters:
I20260809 22:58:39.119685 25643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5519c1e91a5437d88c6f91003e33fa6" candidate_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66957ead7ac94e6e9a377aa45a9c8202" is_pre_election: true
I20260809 22:58:39.120124 25869 raft_consensus.cc:2781] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:58:39.120131 25643 raft_consensus.cc:2454] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd0481bf97b947ff87ce8fa724b49bd4 in term 0.
I20260809 22:58:39.120246 25869 raft_consensus.cc:482] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:58:39.120383 25869 raft_consensus.cc:3037] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:39.122836 25869 raft_consensus.cc:504] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } }
I20260809 22:58:39.123494 25869 leader_election.cc:288] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [CANDIDATE]: Term 1 election: Requested vote from peers 66957ead7ac94e6e9a377aa45a9c8202 (127.24.31.194:42069), f3c0fc40e8ae4a20a81c1219b17672a9 (127.24.31.193:43225)
I20260809 22:58:39.123834 25643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5519c1e91a5437d88c6f91003e33fa6" candidate_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66957ead7ac94e6e9a377aa45a9c8202"
I20260809 22:58:39.123978 25492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5519c1e91a5437d88c6f91003e33fa6" candidate_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9"
I20260809 22:58:39.124034 25643 raft_consensus.cc:3037] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:39.124199 25492 raft_consensus.cc:3037] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:39.126488 25643 raft_consensus.cc:2454] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd0481bf97b947ff87ce8fa724b49bd4 in term 1.
I20260809 22:58:39.126582 25492 raft_consensus.cc:2454] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd0481bf97b947ff87ce8fa724b49bd4 in term 1.
I20260809 22:58:39.126951 25717 leader_election.cc:302] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66957ead7ac94e6e9a377aa45a9c8202, bd0481bf97b947ff87ce8fa724b49bd4; no voters:
I20260809 22:58:39.127287 25869 raft_consensus.cc:2781] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:39.127779 25869 raft_consensus.cc:686] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [term 1 LEADER]: Becoming Leader. State: Replica: bd0481bf97b947ff87ce8fa724b49bd4, State: Running, Role: LEADER
I20260809 22:58:39.128252 25869 consensus_queue.cc:237] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } }
I20260809 22:58:39.133103 25331 catalog_manager.cc:5495] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd0481bf97b947ff87ce8fa724b49bd4 (127.24.31.195). New cstate: current_term: 1 leader_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66957ead7ac94e6e9a377aa45a9c8202" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd0481bf97b947ff87ce8fa724b49bd4" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 33425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c0fc40e8ae4a20a81c1219b17672a9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43225 } health_report { overall_health: UNKNOWN } } }
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:58:39.166010 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.150401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38848 (local address 127.24.31.193:43225)
0809 22:58:39.150602 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.150606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.151258 (+ 652us) server_negotiation.cc:407] Connection header received
0809 22:58:39.151441 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.151446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.151519 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.151653 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:39.153282 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.154044 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.155256 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.155651 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.155774 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.159738 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:39.159754 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:39.159770 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:39.159811 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:39.163167 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.163996 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.164003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.164005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.164154 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.164481 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.164484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.164486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.164960 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:39.165237 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.165470 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.165626 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 22:58:39.167501 25842 tablet.cc:2369] T f5519c1e91a5437d88c6f91003e33fa6 P bd0481bf97b947ff87ce8fa724b49bd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:58:39.172623 25544 tablet.cc:2369] T f5519c1e91a5437d88c6f91003e33fa6 P f3c0fc40e8ae4a20a81c1219b17672a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:58:39.179287 25693 tablet.cc:2369] T f5519c1e91a5437d88c6f91003e33fa6 P 66957ead7ac94e6e9a377aa45a9c8202: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316319, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316319, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:58:39.217298 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.202217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38862 (local address 127.24.31.193:43225)
0809 22:58:39.202482 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.202491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.202805 (+ 314us) server_negotiation.cc:407] Connection header received
0809 22:58:39.203057 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.203061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.203131 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.203252 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:39.204762 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.205639 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.207024 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.207462 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.207617 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.211907 (+ 4290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:39.211913 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:39.211917 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:39.211954 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:39.214731 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.215339 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.215346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.215354 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.215470 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.215846 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.215852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.215855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.216114 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:39.216383 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.216682 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.216915 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:58:39.265184 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.250996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38864 (local address 127.24.31.193:43225)
0809 22:58:39.251219 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.251226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.251410 (+ 184us) server_negotiation.cc:407] Connection header received
0809 22:58:39.251631 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.251635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.251698 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.251813 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:39.253220 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.253928 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.255171 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.255607 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.255742 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.259615 (+ 3873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:39.259621 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:39.259625 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:39.259657 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:39.262380 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.263076 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.263083 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.263086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.263240 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.263614 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.263619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.263623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.263993 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:39.264252 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.264551 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.264791 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 22:58:39.267050 25447 tablet_service.cc:2908] Rejecting scan request for tablet f5519c1e91a5437d88c6f91003e33fa6: Uninitialized: clean time has not yet been initialized
I20260809 22:58:39.270645 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 25398
I20260809 22:58:39.282781 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.271579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.24.31.193:43225)
0809 22:58:39.271734 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.271738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.272070 (+ 332us) server_negotiation.cc:407] Connection header received
0809 22:58:39.272187 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.272191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.272253 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.272330 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:39.273431 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.274109 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.275229 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.275634 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.275730 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.277892 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:39.277896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:39.277899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:39.277926 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:39.280492 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.281075 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.281078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.281080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.281237 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.281612 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.281617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.281619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.281933 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:39.282175 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.282396 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.282564 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:58:39.283612 25537 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:39.376219 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 25398
I20260809 22:58:39.397225 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 25549
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 22:58:39.412287 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.398066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55442 (local address 127.24.31.194:42069)
0809 22:58:39.398247 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.398251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.398269 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:58:39.398362 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.398366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.398429 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.398516 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:39.399713 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.400472 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.401975 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.402383 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.402492 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.406554 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:39.406570 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:39.406587 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:39.406624 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:39.409428 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.410039 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.410046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.410048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.410173 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.410487 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.410490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.410492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.410839 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:39.411259 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.411664 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.411935 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 22:58:39.413000 25687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:39.501793 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 25549
I20260809 22:58:39.521544 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 25697
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 22:58:39.536204 25858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.522364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.24.31.195:33425)
0809 22:58:39.522558 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:39.522562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:39.522581 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:58:39.522729 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:39.522733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:39.522805 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:39.522895 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:39.524152 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.524837 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.526228 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:39.526666 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:39.526761 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:39.530962 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:39.530978 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:39.530996 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:39.531034 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:39.533770 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.534401 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.534409 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.534411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.534535 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:39.534866 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:39.534869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:39.534871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:39.535222 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:39.535436 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:39.535671 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 22:58:39.535849 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:58:39.536929 25836 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:39.630039 25892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.628675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:42069 (local address 127.24.31.195:48331)
0809 22:58:39.629451 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:39.629650 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.194:42069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":567,"threads_started":1}
W20260809 22:58:39.630347 25717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.31.194:42069: connect: Connection refused (error 111)
I20260809 22:58:39.630668 25890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:39.628683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:43225 (local address 127.24.31.195:55195)
0809 22:58:39.630226 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:39.630394 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:43225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1413,"mutex_wait_us":3,"thread_start_us":374,"threads_started":1}
I20260809 22:58:39.631058 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 25697
I20260809 22:58:39.651659 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 25298
I20260809 22:58:39.652932 25366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:39.764269 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 25298
2026-08-09T22:58:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4216 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25897](info): set up 2 sockets
Aug 09 22:58:39 dist-test-slave-vxj2 krb5kdc[25897](info): commencing operation
krb5kdc: starting...
W20260809 22:58:40.918051 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:58:41 dist-test-slave-vxj2 krb5kdc[25897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:58:41.120393 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39269
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:41.459875 25914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:41.460278 25914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:41.460448 25914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:41.467821 25914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:41.467969 25914 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:58:41.468066 25914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:41.468190 25914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:41.468277 25914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:41.468369 25914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:41.478302 25914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39269
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:41.479686 25914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:41.481061 25914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:41.493391 25919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:41.493422 25922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:41.493455 25920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:41.494136 25914 server_base.cc:1034] running on GCE node
I20260809 22:58:41.494853 25914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:41.496364 25914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:41.497575 25914 hybrid_clock.cc:648] HybridClock initialized: now 1786316321497518 us; error 29 us; skew 500 ppm
I20260809 22:58:41.497957 25914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:41 dist-test-slave-vxj2 krb5kdc[25897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316321, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:41.501087 25914 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:58:41.502629 25914 webserver.cc:460] Webserver started at http://127.24.31.254:46035/ using document root <none> and password file <none>
I20260809 22:58:41.503289 25914 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:41.503377 25914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:41.503638 25914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:41.506619 25914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "40dc3485d8d04cd1ab75b32e0e71c088"
format_stamp: "Formatted at 2026-08-09 22:58:41 on dist-test-slave-vxj2"
server_key: "c07b9c26a7d71d3835c6c0f2d21ec8cc"
server_key_iv: "0edcc4610875c94e89146d5ae9236e22"
server_key_version: "encryptionkey@0"
I20260809 22:58:41.507355 25914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "40dc3485d8d04cd1ab75b32e0e71c088"
format_stamp: "Formatted at 2026-08-09 22:58:41 on dist-test-slave-vxj2"
server_key: "c07b9c26a7d71d3835c6c0f2d21ec8cc"
server_key_iv: "0edcc4610875c94e89146d5ae9236e22"
server_key_version: "encryptionkey@0"
I20260809 22:58:41.512588 25914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:58:41.516121 25929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:41.517530 25914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:41.517714 25914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "40dc3485d8d04cd1ab75b32e0e71c088"
format_stamp: "Formatted at 2026-08-09 22:58:41 on dist-test-slave-vxj2"
server_key: "c07b9c26a7d71d3835c6c0f2d21ec8cc"
server_key_iv: "0edcc4610875c94e89146d5ae9236e22"
server_key_version: "encryptionkey@0"
I20260809 22:58:41.517904 25914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:41.535224 25914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:41.539351 25914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:41.539636 25914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:41.562690 25914 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:39269
I20260809 22:58:41.562739 25995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:39269 every 8 connection(s)
I20260809 22:58:41.564103 25914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:41.568396 25996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:41.570030 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 25914
I20260809 22:58:41.570313 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:58:41.571275 24703 external_mini_cluster.cc:1428] Setting key ea51b60c8dfd37121fecead8f834e2e6
I20260809 22:58:41.584942 25996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088: Bootstrap starting.
Aug 09 22:58:41 dist-test-slave-vxj2 krb5kdc[25897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:41.589073 25996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:41.590442 25996 log.cc:824] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:41.593587 25996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088: No bootstrap required, opened a new log
I20260809 22:58:41.595767 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:41.573506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.24.31.254:39269)
0809 22:58:41.574361 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:41.574372 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:41.574430 (+ 58us) server_negotiation.cc:407] Connection header received
0809 22:58:41.576113 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:41.576134 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:41.576512 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:41.576831 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:41.578996 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:41.580257 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:41.582244 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:41.582907 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:41.583049 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:41.588357 (+ 5308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:41.588365 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:41.588379 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:41.588420 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:41.591687 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:41.592401 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:41.592407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:41.592414 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:41.592580 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:41.592955 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:41.592964 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:41.592966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:41.593544 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:41.593853 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:41.594294 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 22:58:41.594763 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":420,"threads_started":1}
I20260809 22:58:41.602464 25996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40dc3485d8d04cd1ab75b32e0e71c088" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39269 } }
I20260809 22:58:41.602875 25996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:41.603010 25996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40dc3485d8d04cd1ab75b32e0e71c088, State: Initialized, Role: FOLLOWER
I20260809 22:58:41.603662 25996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40dc3485d8d04cd1ab75b32e0e71c088" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39269 } }
I20260809 22:58:41.603964 25996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:41.604138 25996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:41.604337 25996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:41.606829 25996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40dc3485d8d04cd1ab75b32e0e71c088" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39269 } }
I20260809 22:58:41.607384 25996 leader_election.cc:302] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40dc3485d8d04cd1ab75b32e0e71c088; no voters:
I20260809 22:58:41.608171 25996 leader_election.cc:288] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:41.608278 26003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:41.609182 26003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [term 1 LEADER]: Becoming Leader. State: Replica: 40dc3485d8d04cd1ab75b32e0e71c088, State: Running, Role: LEADER
I20260809 22:58:41.609807 26003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40dc3485d8d04cd1ab75b32e0e71c088" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39269 } }
I20260809 22:58:41.610647 25996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:41.612917 26004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40dc3485d8d04cd1ab75b32e0e71c088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40dc3485d8d04cd1ab75b32e0e71c088" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39269 } } }
I20260809 22:58:41.612905 26005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40dc3485d8d04cd1ab75b32e0e71c088. Latest consensus state: current_term: 1 leader_uuid: "40dc3485d8d04cd1ab75b32e0e71c088" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40dc3485d8d04cd1ab75b32e0e71c088" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39269 } } }
I20260809 22:58:41.613488 26004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:41.613518 26005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:41.614609 26010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:41.621870 26010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:41.632135 26010 catalog_manager.cc:1347] Generated new cluster ID: dad603cb459548d3a9408d1dac06712d
I20260809 22:58:41.632308 26010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:41.653031 26010 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:41.654415 26010 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:41.665848 26010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088: Generated new TSK 0
I20260809 22:58:41.667155 26010 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:58:41.732366 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39269
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:42.067874 26027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:42.068181 26027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:42.068292 26027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:42.075472 26027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:42.075601 26027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:42.075686 26027 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:58:42.075765 26027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:58:42.085440 26027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.31.254:39269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:42.086665 26027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:42.088014 26027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:42.101934 26032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:42.101898 26033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:42.101974 26035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:42.102849 26027 server_base.cc:1034] running on GCE node
I20260809 22:58:42.103649 26027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:42.104557 26027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:42.105801 26027 hybrid_clock.cc:648] HybridClock initialized: now 1786316322105726 us; error 54 us; skew 500 ppm
I20260809 22:58:42.106110 26027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:42 dist-test-slave-vxj2 krb5kdc[25897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316322, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:42.109386 26027 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:58:42.110913 26027 webserver.cc:460] Webserver started at http://127.24.31.193:44709/ using document root <none> and password file <none>
I20260809 22:58:42.111588 26027 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:42.111673 26027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:42.112054 26027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:42.115041 26027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "7bdd1885cb88449a95fafbda76820ab7"
format_stamp: "Formatted at 2026-08-09 22:58:42 on dist-test-slave-vxj2"
server_key: "f2a647a84b4dc597514b8b1df564e388"
server_key_iv: "54f756e113415b97d81d7553199808f4"
server_key_version: "encryptionkey@0"
I20260809 22:58:42.115840 26027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "7bdd1885cb88449a95fafbda76820ab7"
format_stamp: "Formatted at 2026-08-09 22:58:42 on dist-test-slave-vxj2"
server_key: "f2a647a84b4dc597514b8b1df564e388"
server_key_iv: "54f756e113415b97d81d7553199808f4"
server_key_version: "encryptionkey@0"
I20260809 22:58:42.120801 26027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:58:42.124532 26044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:42.126236 26027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:42.126384 26027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "7bdd1885cb88449a95fafbda76820ab7"
format_stamp: "Formatted at 2026-08-09 22:58:42 on dist-test-slave-vxj2"
server_key: "f2a647a84b4dc597514b8b1df564e388"
server_key_iv: "54f756e113415b97d81d7553199808f4"
server_key_version: "encryptionkey@0"
I20260809 22:58:42.126591 26027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:42.151798 26027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:42.156148 26027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:42.156385 26027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:42.159128 26027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:42.159245 26027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:42.159379 26027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:42.159449 26027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:58:42 dist-test-slave-vxj2 krb5kdc[25897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316322, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:42.185353 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:42.159882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:35073 (local address 127.24.31.254:39269)
0809 22:58:42.160438 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:42.160442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:42.161656 (+ 1214us) server_negotiation.cc:407] Connection header received
0809 22:58:42.163702 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:42.163707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:42.163810 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:42.163947 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:42.166612 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.167677 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.171126 (+ 3449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.171613 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.171765 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:42.177244 (+ 5479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:42.177250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:42.177253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:42.177289 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:42.180880 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:42.182152 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:42.182158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:42.182161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:42.182299 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:42.182918 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:42.182923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:42.182927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:42.183335 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:42.183489 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:42.184422 (+ 933us) server_negotiation.cc:299] Negotiation successful
0809 22:58:42.184786 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":321,"threads_started":1}
I20260809 22:58:42.186084 26054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:42.160585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39269 (local address 127.24.31.193:35073)
0809 22:58:42.161492 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:42.161535 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:42.163410 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:42.164280 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:42.164295 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:42.165383 (+ 1088us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:42.166376 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:42.166407 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.167869 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.167884 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:42.170947 (+ 3063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:42.170969 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.171921 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.171932 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:42.172337 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:42.173353 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:42.173388 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:42.176927 (+ 3539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:42.181054 (+ 4127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:42.181065 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:42.181096 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:42.181986 (+ 890us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:42.182432 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:42.182437 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:42.182441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:42.182802 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:42.183559 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:42.183565 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:42.183941 (+ 376us) client_negotiation.cc:769] Sending connection context
0809 22:58:42.184191 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 22:58:42.184742 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":362,"threads_started":1}
I20260809 22:58:42.207711 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:42.191839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:44627 (local address 127.24.31.254:39269)
0809 22:58:42.192015 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:42.192022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:42.192745 (+ 723us) server_negotiation.cc:407] Connection header received
0809 22:58:42.192848 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:42.192856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:42.192960 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:42.193083 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:42.194537 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.195530 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.197788 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.198367 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.198518 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:42.201347 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:42.201354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:42.201359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:42.201404 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:42.204332 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:42.205351 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:42.205357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:42.205360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:42.205542 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:42.206042 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:42.206047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:42.206051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:42.206459 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:42.206647 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:42.207086 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 22:58:42.207325 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:58:42.207984 26115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:42.191922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39269 (local address 127.24.31.193:44627)
0809 22:58:42.192627 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:42.192643 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:42.192752 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:42.193114 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:42.193116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:42.193821 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:42.194387 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:42.194403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.195684 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.195689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:42.197645 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:42.197659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.198579 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.198585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:42.198761 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:42.199464 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:42.199495 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:42.201073 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:42.204469 (+ 3396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:42.204478 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:42.204483 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:42.205210 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:42.205731 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:42.205737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:42.205743 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:42.205910 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:42.206611 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:42.206618 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:42.206824 (+ 206us) client_negotiation.cc:769] Sending connection context
0809 22:58:42.206981 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 22:58:42.207420 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":406,"threads_started":1}
I20260809 22:58:42.215284 26027 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:39635
I20260809 22:58:42.215291 26175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:39635 every 8 connection(s)
I20260809 22:58:42.216500 26027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:58:42.218561 26176 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39269
I20260809 22:58:42.218802 26176 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:42.219403 26176 heartbeater.cc:511] Master 127.24.31.254:39269 requested a full tablet report, sending...
I20260809 22:58:42.221349 25947 ts_manager.cc:194] Registered new tserver with Master: 7bdd1885cb88449a95fafbda76820ab7 (127.24.31.193:39635)
I20260809 22:58:42.223016 25947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:35073
I20260809 22:58:42.225663 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 26027
I20260809 22:58:42.226092 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:58:42.226842 24703 external_mini_cluster.cc:1428] Setting key d88c6d826167efbd7b61a137df4ec9a2
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:58:42.270545 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39269
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:42.605957 26180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:42.606256 26180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:42.606462 26180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:42.613848 26180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:42.613982 26180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:42.614142 26180 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:58:42.614271 26180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:58:42.624444 26180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.31.254:39269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:42.625790 26180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:42.627135 26180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:42.640262 26186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:42.640362 26189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:42.640229 26187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:42.641428 26180 server_base.cc:1034] running on GCE node
I20260809 22:58:42.642063 26180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:42.642982 26180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:42.644197 26180 hybrid_clock.cc:648] HybridClock initialized: now 1786316322644139 us; error 32 us; skew 500 ppm
I20260809 22:58:42.644500 26180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:42 dist-test-slave-vxj2 krb5kdc[25897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316322, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:42.651650 26180 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:58:42.653257 26180 webserver.cc:460] Webserver started at http://127.24.31.194:32803/ using document root <none> and password file <none>
I20260809 22:58:42.653981 26180 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:42.654111 26180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:42.654469 26180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:42.657371 26180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "db161bf35bab42f18ef772519f6fb774"
format_stamp: "Formatted at 2026-08-09 22:58:42 on dist-test-slave-vxj2"
server_key: "d93fa5e9862721f039b7ae3738906b9b"
server_key_iv: "2e00e85dd5cc0e8e23388fefd0bf963c"
server_key_version: "encryptionkey@0"
I20260809 22:58:42.658193 26180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "db161bf35bab42f18ef772519f6fb774"
format_stamp: "Formatted at 2026-08-09 22:58:42 on dist-test-slave-vxj2"
server_key: "d93fa5e9862721f039b7ae3738906b9b"
server_key_iv: "2e00e85dd5cc0e8e23388fefd0bf963c"
server_key_version: "encryptionkey@0"
I20260809 22:58:42.663326 26180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:58:42.666770 26199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:42.668083 26180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:58:42.668262 26180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "db161bf35bab42f18ef772519f6fb774"
format_stamp: "Formatted at 2026-08-09 22:58:42 on dist-test-slave-vxj2"
server_key: "d93fa5e9862721f039b7ae3738906b9b"
server_key_iv: "2e00e85dd5cc0e8e23388fefd0bf963c"
server_key_version: "encryptionkey@0"
I20260809 22:58:42.668496 26180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:42.690213 26180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:42.695120 26180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:42.695418 26180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:42.698426 26180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:42.698542 26180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:42.698680 26180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:42.698751 26180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:58:42 dist-test-slave-vxj2 krb5kdc[25897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316322, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:42.726055 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:42.699198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:37605 (local address 127.24.31.254:39269)
0809 22:58:42.699436 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:42.699443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:42.701062 (+ 1619us) server_negotiation.cc:407] Connection header received
0809 22:58:42.704119 (+ 3057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:42.704126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:42.704238 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:42.704381 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:42.706894 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.708038 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.710615 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.711385 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.711551 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:42.717122 (+ 5571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:42.717127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:42.717131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:42.717249 (+ 118us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:42.720971 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:42.722327 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:42.722334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:42.722338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:42.722506 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:42.723273 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:42.723277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:42.723279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:42.723715 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:42.723894 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:42.725356 (+ 1462us) server_negotiation.cc:299] Negotiation successful
0809 22:58:42.725597 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 22:58:42.727959 26208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:42.699875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39269 (local address 127.24.31.194:37605)
0809 22:58:42.700866 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:42.700920 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:42.703741 (+ 2821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:42.704660 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:42.704668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:42.705538 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:42.706646 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:42.706682 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.708218 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.708225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:42.710311 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:42.710398 (+ 87us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.711690 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.711699 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:42.712046 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:42.713088 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:42.713138 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:42.716847 (+ 3709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:42.721220 (+ 4373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:42.721241 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:42.721253 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:42.722075 (+ 822us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:42.722666 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:42.722671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:42.722675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:42.723071 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:42.723983 (+ 912us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:42.723989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:42.724565 (+ 576us) client_negotiation.cc:769] Sending connection context
0809 22:58:42.724954 (+ 389us) client_negotiation.cc:240] Negotiation successful
0809 22:58:42.725615 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":407,"threads_started":1}
I20260809 22:58:42.741230 26180 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:43451
I20260809 22:58:42.741273 26332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:43451 every 8 connection(s)
I20260809 22:58:42.742465 26180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:58:42.745518 26333 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39269
I20260809 22:58:42.745735 26333 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:42.746649 26333 heartbeater.cc:511] Master 127.24.31.254:39269 requested a full tablet report, sending...
I20260809 22:58:42.748320 25947 ts_manager.cc:194] Registered new tserver with Master: db161bf35bab42f18ef772519f6fb774 (127.24.31.194:43451)
I20260809 22:58:42.749238 25947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:37605
I20260809 22:58:42.751547 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 26180
I20260809 22:58:42.751749 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:58:42.751871 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:42.736382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:39109 (local address 127.24.31.254:39269)
0809 22:58:42.736593 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:42.736601 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:42.736901 (+ 300us) server_negotiation.cc:407] Connection header received
0809 22:58:42.737226 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:42.737232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:42.737310 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:42.737421 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:42.739033 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.739858 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.741830 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.742282 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.742446 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:42.744909 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:42.744914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:42.744918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:42.744955 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:42.748616 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:42.749381 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:42.749388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:42.749392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:42.749580 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:42.750236 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:42.750242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:42.750245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:42.750701 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:42.750888 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:42.751261 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 22:58:42.751487 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:58:42.752036 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:42.735906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39269 (local address 127.24.31.194:39109)
0809 22:58:42.736782 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:42.736807 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:42.737029 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:42.737455 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:42.737464 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:42.738164 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:42.738780 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:42.738801 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.740123 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.740131 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:42.741666 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:42.741681 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:42.742500 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:42.742507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:42.742714 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:42.743438 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:42.743461 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:42.744704 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:42.748766 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:42.748773 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:42.748777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:42.749247 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:42.749988 (+ 741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:42.749991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:42.749994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:42.750114 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:42.750841 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:42.750844 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:42.751007 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 22:58:42.751146 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 22:58:42.751503 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":679,"threads_started":1}
I20260809 22:58:42.752382 24703 external_mini_cluster.cc:1428] Setting key f3158fc3ac0d0bda139d841d12ba41b1
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:58:42.796185 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39269
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:43.115973 26337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:43.116245 26337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:43.116426 26337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:43.123780 26337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:43.123910 26337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:43.124073 26337 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:58:43.124193 26337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:58:43.134002 26337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.31.254:39269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:43.135293 26337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:43.136621 26337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:43.149551 26342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:43.149581 26346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:43.149590 26343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:43.150547 26337 server_base.cc:1034] running on GCE node
I20260809 22:58:43.151144 26337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:43.152050 26337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:43.153265 26337 hybrid_clock.cc:648] HybridClock initialized: now 1786316323153190 us; error 55 us; skew 500 ppm
I20260809 22:58:43.153522 26337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:43 dist-test-slave-vxj2 krb5kdc[25897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316323, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:43.156628 26337 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:58:43.158090 26337 webserver.cc:460] Webserver started at http://127.24.31.195:35961/ using document root <none> and password file <none>
I20260809 22:58:43.158793 26337 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:43.158929 26337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:43.159250 26337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:43.162271 26337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "2269eb7f873942cd980b7cd5189dbcfa"
format_stamp: "Formatted at 2026-08-09 22:58:43 on dist-test-slave-vxj2"
server_key: "a0bef826fe304639a031fb1fec5a2ccb"
server_key_iv: "df501950c78d5921a29a5e4657daac01"
server_key_version: "encryptionkey@0"
I20260809 22:58:43.163179 26337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "2269eb7f873942cd980b7cd5189dbcfa"
format_stamp: "Formatted at 2026-08-09 22:58:43 on dist-test-slave-vxj2"
server_key: "a0bef826fe304639a031fb1fec5a2ccb"
server_key_iv: "df501950c78d5921a29a5e4657daac01"
server_key_version: "encryptionkey@0"
I20260809 22:58:43.168318 26337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 22:58:43.171792 26354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:43.173224 26337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:43.173394 26337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "2269eb7f873942cd980b7cd5189dbcfa"
format_stamp: "Formatted at 2026-08-09 22:58:43 on dist-test-slave-vxj2"
server_key: "a0bef826fe304639a031fb1fec5a2ccb"
server_key_iv: "df501950c78d5921a29a5e4657daac01"
server_key_version: "encryptionkey@0"
I20260809 22:58:43.173609 26337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:43.203688 26337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:43.208130 26337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:43.208442 26337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:43.211372 26337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:43.211480 26337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:43.211623 26337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:43.211711 26337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:43.225693 26176 heartbeater.cc:503] Master 127.24.31.254:39269 was elected leader, sending a full tablet report...
Aug 09 22:58:43 dist-test-slave-vxj2 krb5kdc[25897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316323, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:43.236254 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.211882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:42489 (local address 127.24.31.254:39269)
0809 22:58:43.212116 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.212124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.213598 (+ 1474us) server_negotiation.cc:407] Connection header received
0809 22:58:43.215655 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.215661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.215762 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.215873 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:43.218297 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.219540 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.222637 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.223329 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.223480 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.227965 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:43.227971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:43.227974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:43.228004 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:43.232485 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:43.233294 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:43.233303 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:43.233310 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:43.233491 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:43.234122 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:43.234127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:43.234131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:43.234548 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:43.234715 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.235636 (+ 921us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.235867 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:58:43.237552 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.212588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39269 (local address 127.24.31.195:42489)
0809 22:58:43.213423 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.213471 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.215367 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.216231 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.216239 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.217016 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:43.218039 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.218072 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.219751 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.219758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.222438 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.222456 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.223600 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.223608 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.223806 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.224587 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:43.224612 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:43.227705 (+ 3093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:43.232634 (+ 4929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:43.232641 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:43.232652 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:43.233097 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:43.233643 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:43.233651 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:43.233657 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:43.233966 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:43.234695 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:43.234702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:43.235230 (+ 528us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.235523 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.236107 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":343,"threads_started":1}
I20260809 22:58:43.258556 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.243015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:50883 (local address 127.24.31.254:39269)
0809 22:58:43.243195 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.243202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.244054 (+ 852us) server_negotiation.cc:407] Connection header received
0809 22:58:43.244267 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.244272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.244372 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.244479 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:43.245971 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.246653 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.249143 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.249756 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.249897 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.252032 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:43.252040 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:43.252047 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:43.252104 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:43.255807 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:43.256460 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:43.256467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:43.256473 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:43.256604 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:43.256973 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:43.256978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:43.256981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:43.257315 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:43.257481 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.257930 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.258182 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:58:43.258729 26440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.243074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39269 (local address 127.24.31.195:50883)
0809 22:58:43.243949 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.243966 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.244129 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.244523 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.244530 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.245302 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:43.245798 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.245815 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.246801 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.246810 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.248991 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.249011 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.249969 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.249977 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.250180 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.250858 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:43.250887 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:43.251824 (+ 937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:43.255946 (+ 4122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:43.255952 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:43.255954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:43.256339 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:43.256721 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:43.256727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:43.256731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:43.256855 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:43.257440 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:43.257452 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:43.257664 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.257830 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.258210 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":515,"threads_started":1}
I20260809 22:58:43.270545 26337 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:37065
I20260809 22:58:43.270571 26491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:37065 every 8 connection(s)
I20260809 22:58:43.271876 26337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:58:43.274135 26492 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39269
I20260809 22:58:43.274326 26492 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:43.274900 26492 heartbeater.cc:511] Master 127.24.31.254:39269 requested a full tablet report, sending...
I20260809 22:58:43.275787 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 26337
I20260809 22:58:43.276000 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:58:43.276265 25947 ts_manager.cc:194] Registered new tserver with Master: 2269eb7f873942cd980b7cd5189dbcfa (127.24.31.195:37065)
I20260809 22:58:43.276548 24703 external_mini_cluster.cc:1428] Setting key 8a94d20cd41a6c138a1bd135c67006e1
I20260809 22:58:43.277263 25947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:42489
I20260809 22:58:43.279781 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:58:43 dist-test-slave-vxj2 krb5kdc[25897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316323, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:58:43 dist-test-slave-vxj2 krb5kdc[25897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316323, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:43.328593 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.314174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.24.31.254:39269)
0809 22:58:43.314352 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.314357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.314776 (+ 419us) server_negotiation.cc:407] Connection header received
0809 22:58:43.314941 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.314945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.315012 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.315103 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:43.316630 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.317534 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.318873 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.319291 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.319381 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.323812 (+ 4431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:43.323817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:43.323820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:43.323854 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:43.326404 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:43.327029 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:43.327033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:43.327035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:43.327164 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:43.327489 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:43.327492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:43.327494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:43.327744 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:43.327924 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.328148 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.328311 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:58:43.332788 25947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:58:43.339704 25947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:58:43.373070 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.352571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:37065 (local address 127.0.0.1:50330)
0809 22:58:43.353706 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.353751 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.353920 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.361954 (+ 8034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.361966 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.362009 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:43.362564 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.362580 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.365271 (+ 2691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.365280 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.368573 (+ 3293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.368595 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.371046 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.371055 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.371348 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.371426 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.371549 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.371839 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":31,"thread_start_us":699,"threads_started":1}
I20260809 22:58:43.373848 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.353538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50330 (local address 127.24.31.195:37065)
0809 22:58:43.361174 (+ 7636us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.361186 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.361228 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:58:43.361355 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.361363 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.361635 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.362830 (+ 1195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:43.363221 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.365054 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.368808 (+ 3754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.370729 (+ 1921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.372424 (+ 1695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.372706 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.372814 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.373042 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7420,"thread_start_us":7248,"threads_started":1}
I20260809 22:58:43.373883 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.353444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49114 (local address 127.24.31.193:39635)
0809 22:58:43.360584 (+ 7140us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.360593 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.361850 (+ 1257us) server_negotiation.cc:407] Connection header received
0809 22:58:43.361967 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.361974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.362192 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.362424 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:43.363575 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.365623 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.369624 (+ 4001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.371624 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.371812 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.372095 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.373380 (+ 1285us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.373552 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6972,"thread_start_us":6831,"threads_started":1}
I20260809 22:58:43.374691 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.352866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:39635 (local address 127.0.0.1:49114)
0809 22:58:43.361528 (+ 8662us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.361543 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.361697 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.362729 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.362737 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.362785 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:43.363326 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.363354 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.365844 (+ 2490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.365852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.369403 (+ 3551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.369420 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.372823 (+ 3403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.372834 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.373117 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.373175 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.374123 (+ 948us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.374269 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8552,"mutex_wait_us":1,"thread_start_us":8428,"threads_started":1}
I20260809 22:58:43.378450 26413 tablet_service.cc:1467] Processing CreateTablet for tablet 381e86abccd64f3e94016ed4e47b291b (DEFAULT_TABLE table=test-table [id=12668c08dfeb404bab586a0ef9cbb2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:43.378445 26098 tablet_service.cc:1467] Processing CreateTablet for tablet 381e86abccd64f3e94016ed4e47b291b (DEFAULT_TABLE table=test-table [id=12668c08dfeb404bab586a0ef9cbb2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:43.379750 26413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381e86abccd64f3e94016ed4e47b291b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:43.379792 26098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381e86abccd64f3e94016ed4e47b291b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:43.382664 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.361693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:43451 (local address 127.0.0.1:43744)
0809 22:58:43.373579 (+ 11886us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.373596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.373718 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.374688 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.374693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.374721 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:43.374908 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.374923 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.378087 (+ 3164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.378092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.379943 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.379958 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.381919 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.381926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.382106 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.382140 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.382235 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.382392 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11800,"thread_start_us":13569,"threads_started":1}
I20260809 22:58:43.383562 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.361976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43744 (local address 127.24.31.194:43451)
0809 22:58:43.372662 (+ 10686us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.372670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.373846 (+ 1176us) server_negotiation.cc:407] Connection header received
0809 22:58:43.373963 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.373975 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.374255 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.374451 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:43.375112 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.377933 (+ 2821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.380142 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.381477 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.381621 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.381815 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.383029 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.383211 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10528,"thread_start_us":10403,"threads_started":1}
I20260809 22:58:43.387398 26252 tablet_service.cc:1467] Processing CreateTablet for tablet 381e86abccd64f3e94016ed4e47b291b (DEFAULT_TABLE table=test-table [id=12668c08dfeb404bab586a0ef9cbb2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:43.388808 26252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381e86abccd64f3e94016ed4e47b291b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:43.395817 26512 tablet_bootstrap.cc:492] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: Bootstrap starting.
I20260809 22:58:43.399396 26513 tablet_bootstrap.cc:492] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: Bootstrap starting.
I20260809 22:58:43.399947 26512 tablet_bootstrap.cc:654] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:43.401350 26512 log.cc:824] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:43.401592 26514 tablet_bootstrap.cc:492] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: Bootstrap starting.
I20260809 22:58:43.402742 26513 tablet_bootstrap.cc:654] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:43.404009 26513 log.cc:824] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:43.405304 26514 tablet_bootstrap.cc:654] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:43.406216 26512 tablet_bootstrap.cc:492] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: No bootstrap required, opened a new log
I20260809 22:58:43.406452 26514 log.cc:824] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:43.406543 26512 ts_tablet_manager.cc:1397] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260809 22:58:43.407676 26513 tablet_bootstrap.cc:492] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: No bootstrap required, opened a new log
I20260809 22:58:43.407982 26513 ts_tablet_manager.cc:1397] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 22:58:43.409547 26514 tablet_bootstrap.cc:492] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: No bootstrap required, opened a new log
I20260809 22:58:43.409797 26514 ts_tablet_manager.cc:1397] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260809 22:58:43.415264 26512 raft_consensus.cc:348] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } }
I20260809 22:58:43.415666 26512 raft_consensus.cc:374] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:43.415787 26512 raft_consensus.cc:729] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bdd1885cb88449a95fafbda76820ab7, State: Initialized, Role: FOLLOWER
I20260809 22:58:43.416476 26512 consensus_queue.cc:260] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } }
I20260809 22:58:43.418762 26513 raft_consensus.cc:348] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } }
I20260809 22:58:43.419176 26513 raft_consensus.cc:374] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:43.419274 26513 raft_consensus.cc:729] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2269eb7f873942cd980b7cd5189dbcfa, State: Initialized, Role: FOLLOWER
I20260809 22:58:43.419965 26512 ts_tablet_manager.cc:1428] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260809 22:58:43.419978 26513 consensus_queue.cc:260] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } }
I20260809 22:58:43.420289 26514 raft_consensus.cc:348] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } }
I20260809 22:58:43.420749 26514 raft_consensus.cc:374] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:43.420867 26514 raft_consensus.cc:729] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db161bf35bab42f18ef772519f6fb774, State: Initialized, Role: FOLLOWER
I20260809 22:58:43.421415 26492 heartbeater.cc:503] Master 127.24.31.254:39269 was elected leader, sending a full tablet report...
I20260809 22:58:43.421456 26514 consensus_queue.cc:260] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } }
I20260809 22:58:43.422823 26333 heartbeater.cc:503] Master 127.24.31.254:39269 was elected leader, sending a full tablet report...
I20260809 22:58:43.423288 26514 ts_tablet_manager.cc:1428] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260809 22:58:43.424062 26513 ts_tablet_manager.cc:1428] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260809 22:58:43.425824 26520 raft_consensus.cc:482] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:43.426119 26520 raft_consensus.cc:504] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } }
I20260809 22:58:43.427963 26520 leader_election.cc:288] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2269eb7f873942cd980b7cd5189dbcfa (127.24.31.195:37065), 7bdd1885cb88449a95fafbda76820ab7 (127.24.31.193:39635)
I20260809 22:58:43.435917 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.428654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:54199 (local address 127.24.31.195:37065)
0809 22:58:43.429137 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.429172 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.429198 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:58:43.429277 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.429286 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.429381 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.429496 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:43.430240 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.431306 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.433838 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.434835 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.435030 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.435107 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.435525 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.435655 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306}
I20260809 22:58:43.436124 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.428884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:43263 (local address 127.24.31.193:39635)
0809 22:58:43.429067 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.429072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.430346 (+ 1274us) server_negotiation.cc:407] Connection header received
0809 22:58:43.430566 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.430572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.430662 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.430777 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:43.431365 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.432197 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.434384 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.435146 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.435330 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.435412 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.435758 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.435890 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 22:58:43.436319 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.428378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:39635 (local address 127.24.31.194:43263)
0809 22:58:43.430237 (+ 1859us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.430263 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.430426 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.430848 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.430851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.430870 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:43.431195 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.431206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.432338 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.432344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.434222 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.434233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.435387 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.435393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.435588 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.435619 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.435716 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.435853 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1731,"mutex_wait_us":24,"thread_start_us":1530,"threads_started":1}
I20260809 22:58:43.436764 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.428167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:37065 (local address 127.24.31.194:54199)
0809 22:58:43.428895 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.428915 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.429046 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.429543 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.429548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.429587 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:43.430080 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.430095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.431417 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.431421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.433648 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.433663 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.435065 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.435071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.435290 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.435345 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.436140 (+ 795us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.436311 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":19,"thread_start_us":454,"threads_started":1}
I20260809 22:58:43.437456 26125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381e86abccd64f3e94016ed4e47b291b" candidate_uuid: "db161bf35bab42f18ef772519f6fb774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bdd1885cb88449a95fafbda76820ab7" is_pre_election: true
I20260809 22:58:43.437743 26441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381e86abccd64f3e94016ed4e47b291b" candidate_uuid: "db161bf35bab42f18ef772519f6fb774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2269eb7f873942cd980b7cd5189dbcfa" is_pre_election: true
I20260809 22:58:43.438054 26125 raft_consensus.cc:2454] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db161bf35bab42f18ef772519f6fb774 in term 0.
I20260809 22:58:43.438254 26441 raft_consensus.cc:2454] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db161bf35bab42f18ef772519f6fb774 in term 0.
I20260809 22:58:43.438777 26202 leader_election.cc:302] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bdd1885cb88449a95fafbda76820ab7, db161bf35bab42f18ef772519f6fb774; no voters:
I20260809 22:58:43.439159 26520 raft_consensus.cc:2781] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:58:43.439322 26520 raft_consensus.cc:482] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:58:43.439458 26520 raft_consensus.cc:3037] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:43.441725 26520 raft_consensus.cc:504] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } }
I20260809 22:58:43.442281 26520 leader_election.cc:288] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [CANDIDATE]: Term 1 election: Requested vote from peers 2269eb7f873942cd980b7cd5189dbcfa (127.24.31.195:37065), 7bdd1885cb88449a95fafbda76820ab7 (127.24.31.193:39635)
I20260809 22:58:43.442775 26125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381e86abccd64f3e94016ed4e47b291b" candidate_uuid: "db161bf35bab42f18ef772519f6fb774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bdd1885cb88449a95fafbda76820ab7"
I20260809 22:58:43.442811 26441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381e86abccd64f3e94016ed4e47b291b" candidate_uuid: "db161bf35bab42f18ef772519f6fb774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2269eb7f873942cd980b7cd5189dbcfa"
I20260809 22:58:43.442970 26125 raft_consensus.cc:3037] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:43.443071 26441 raft_consensus.cc:3037] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:43.445283 26125 raft_consensus.cc:2454] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db161bf35bab42f18ef772519f6fb774 in term 1.
I20260809 22:58:43.445611 26441 raft_consensus.cc:2454] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db161bf35bab42f18ef772519f6fb774 in term 1.
I20260809 22:58:43.445636 26202 leader_election.cc:302] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bdd1885cb88449a95fafbda76820ab7, db161bf35bab42f18ef772519f6fb774; no voters:
I20260809 22:58:43.445947 26520 raft_consensus.cc:2781] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:43.446486 26520 raft_consensus.cc:686] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 1 LEADER]: Becoming Leader. State: Replica: db161bf35bab42f18ef772519f6fb774, State: Running, Role: LEADER
I20260809 22:58:43.446990 26520 consensus_queue.cc:237] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } }
I20260809 22:58:43.451233 25946 catalog_manager.cc:5495] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 reported cstate change: term changed from 0 to 1, leader changed from <none> to db161bf35bab42f18ef772519f6fb774 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "db161bf35bab42f18ef772519f6fb774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } health_report { overall_health: HEALTHY } } }
W20260809 22:58:43.469257 26177 tablet.cc:2369] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:58:43.474815 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.468885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43758 (local address 127.24.31.194:43451)
0809 22:58:43.469248 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.469253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.469271 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:58:43.469618 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.469622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.469731 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.469843 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:43.470388 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.471126 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.472651 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.473179 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.473288 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.473804 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:43.474300 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:43.474409 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.474540 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.474609 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260809 22:58:43.483690 26125 raft_consensus.cc:1261] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Refusing update from remote peer db161bf35bab42f18ef772519f6fb774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:43.483748 26441 raft_consensus.cc:1261] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [term 1 FOLLOWER]: Refusing update from remote peer db161bf35bab42f18ef772519f6fb774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:43.484716 26523 consensus_queue.cc:1035] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:43.485109 26520 consensus_queue.cc:1035] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:58:43.495168 26334 tablet.cc:2369] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:58:43.497952 26526 mvcc.cc:205] Tried to move back new op lower bound from 7316751660979724288 to 7316751660842995712. Current Snapshot: MvccSnapshot[applied={T|T < 7316751660979724288}]
I20260809 22:58:43.509423 26527 mvcc.cc:205] Tried to move back new op lower bound from 7316751660979724288 to 7316751660842995712. Current Snapshot: MvccSnapshot[applied={T|T < 7316751660979724288}]
I20260809 22:58:43.523564 25946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:43.523897 25946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:43.527925 25946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088: Sending DeleteTablet for 3 replicas of tablet 381e86abccd64f3e94016ed4e47b291b
I20260809 22:58:43.529215 26413 tablet_service.cc:1514] Processing DeleteTablet for tablet 381e86abccd64f3e94016ed4e47b291b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:43 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:50330
I20260809 22:58:43.529587 26098 tablet_service.cc:1514] Processing DeleteTablet for tablet 381e86abccd64f3e94016ed4e47b291b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:43 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:49114
I20260809 22:58:43.530364 26252 tablet_service.cc:1514] Processing DeleteTablet for tablet 381e86abccd64f3e94016ed4e47b291b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:43 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:43744
I20260809 22:58:43.530690 25946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:58:43.531481 25946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:58:43.537968 26540 tablet_replica.cc:330] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: stopping tablet replica
I20260809 22:58:43.538069 26541 tablet_replica.cc:330] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: stopping tablet replica
I20260809 22:58:43.538455 26540 raft_consensus.cc:2229] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:43.538529 26541 raft_consensus.cc:2229] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:58:43.540118 26540 raft_consensus.cc:2258] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:43.540359 26541 raft_consensus.cc:2258] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:43.541501 26252 tablet_service.cc:1467] Processing CreateTablet for tablet 5bca656291e44ecbbb839a3e292091a7 (DEFAULT_TABLE table=test-table [id=a3c7da70759c46a4bf5052035503ad38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:43.541726 26412 tablet_service.cc:1467] Processing CreateTablet for tablet 5bca656291e44ecbbb839a3e292091a7 (DEFAULT_TABLE table=test-table [id=a3c7da70759c46a4bf5052035503ad38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:43.542032 26252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bca656291e44ecbbb839a3e292091a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:43.542357 26412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bca656291e44ecbbb839a3e292091a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:43.544602 26098 tablet_service.cc:1467] Processing CreateTablet for tablet 5bca656291e44ecbbb839a3e292091a7 (DEFAULT_TABLE table=test-table [id=a3c7da70759c46a4bf5052035503ad38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:43.545203 26098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bca656291e44ecbbb839a3e292091a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:43.545778 26540 ts_tablet_manager.cc:1905] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:43.549125 26514 tablet_bootstrap.cc:492] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: Bootstrap starting.
I20260809 22:58:43.550940 26514 tablet_bootstrap.cc:654] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:43.552728 26540 ts_tablet_manager.cc:1918] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:58:43.552891 26540 log.cc:1196] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/381e86abccd64f3e94016ed4e47b291b
I20260809 22:58:43.544467 26541 ts_tablet_manager.cc:1905] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:43.557407 26513 tablet_bootstrap.cc:492] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: Bootstrap starting.
I20260809 22:58:43.559732 26513 tablet_bootstrap.cc:654] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:43.563015 26539 tablet_replica.cc:330] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: stopping tablet replica
I20260809 22:58:43.563460 26539 raft_consensus.cc:2229] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:43.563942 26539 raft_consensus.cc:2258] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:43.564616 26540 ts_tablet_manager.cc:1939] T 381e86abccd64f3e94016ed4e47b291b P 7bdd1885cb88449a95fafbda76820ab7: Deleting consensus metadata
I20260809 22:58:43.566468 25932 catalog_manager.cc:4841] TS 7bdd1885cb88449a95fafbda76820ab7 (127.24.31.193:39635): tablet 381e86abccd64f3e94016ed4e47b291b (table test-table [id=12668c08dfeb404bab586a0ef9cbb2b3]) successfully deleted
I20260809 22:58:43.568804 26539 ts_tablet_manager.cc:1905] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:43.570052 26541 ts_tablet_manager.cc:1918] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:58:43.570237 26541 log.cc:1196] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/381e86abccd64f3e94016ed4e47b291b
I20260809 22:58:43.571103 26541 ts_tablet_manager.cc:1939] T 381e86abccd64f3e94016ed4e47b291b P db161bf35bab42f18ef772519f6fb774: Deleting consensus metadata
I20260809 22:58:43.573180 25932 catalog_manager.cc:4841] TS db161bf35bab42f18ef772519f6fb774 (127.24.31.194:43451): tablet 381e86abccd64f3e94016ed4e47b291b (table test-table [id=12668c08dfeb404bab586a0ef9cbb2b3]) successfully deleted
I20260809 22:58:43.577977 26539 ts_tablet_manager.cc:1918] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:58:43.581498 26512 tablet_bootstrap.cc:492] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: Bootstrap starting.
I20260809 22:58:43.582082 26539 log.cc:1196] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/381e86abccd64f3e94016ed4e47b291b
I20260809 22:58:43.583341 26512 tablet_bootstrap.cc:654] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:43.583366 26514 tablet_bootstrap.cc:492] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: No bootstrap required, opened a new log
I20260809 22:58:43.583559 26514 ts_tablet_manager.cc:1397] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.019s
I20260809 22:58:43.584489 26514 raft_consensus.cc:348] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.584830 26514 raft_consensus.cc:374] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:43.585130 26539 ts_tablet_manager.cc:1939] T 381e86abccd64f3e94016ed4e47b291b P 2269eb7f873942cd980b7cd5189dbcfa: Deleting consensus metadata
I20260809 22:58:43.585302 26514 raft_consensus.cc:729] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db161bf35bab42f18ef772519f6fb774, State: Initialized, Role: FOLLOWER
I20260809 22:58:43.585704 26514 consensus_queue.cc:260] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.587406 26514 ts_tablet_manager.cc:1428] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 22:58:43.590377 25931 catalog_manager.cc:4841] TS 2269eb7f873942cd980b7cd5189dbcfa (127.24.31.195:37065): tablet 381e86abccd64f3e94016ed4e47b291b (table test-table [id=12668c08dfeb404bab586a0ef9cbb2b3]) successfully deleted
I20260809 22:58:43.591041 26512 tablet_bootstrap.cc:492] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: No bootstrap required, opened a new log
I20260809 22:58:43.591068 26513 tablet_bootstrap.cc:492] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: No bootstrap required, opened a new log
I20260809 22:58:43.591194 26513 ts_tablet_manager.cc:1397] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.010s
I20260809 22:58:43.591207 26512 ts_tablet_manager.cc:1397] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.005s
I20260809 22:58:43.591976 26513 raft_consensus.cc:348] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.592043 26512 raft_consensus.cc:348] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.592204 26513 raft_consensus.cc:374] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:43.592283 26512 raft_consensus.cc:374] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:43.592294 26513 raft_consensus.cc:729] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2269eb7f873942cd980b7cd5189dbcfa, State: Initialized, Role: FOLLOWER
I20260809 22:58:43.592381 26512 raft_consensus.cc:729] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bdd1885cb88449a95fafbda76820ab7, State: Initialized, Role: FOLLOWER
I20260809 22:58:43.592562 26513 consensus_queue.cc:260] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.592697 26512 consensus_queue.cc:260] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.593134 26513 ts_tablet_manager.cc:1428] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:58:43.594277 26512 ts_tablet_manager.cc:1428] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:58:43.721446 26519 raft_consensus.cc:482] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:43.721732 26519 raft_consensus.cc:504] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.723330 26519 leader_election.cc:288] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bdd1885cb88449a95fafbda76820ab7 (127.24.31.193:39635), db161bf35bab42f18ef772519f6fb774 (127.24.31.194:43451)
I20260809 22:58:43.730602 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.723853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:48835 (local address 127.24.31.193:39635)
0809 22:58:43.724038 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.724045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.724069 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:58:43.724329 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.724335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.724412 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.724509 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:43.725194 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.726232 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.728167 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.729514 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.729692 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.729769 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.730102 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.730244 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:58:43.730607 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.723768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:39635 (local address 127.24.31.195:48835)
0809 22:58:43.723970 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.723991 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.724141 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.724706 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.724711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.724745 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:43.724970 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.724985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.726365 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.726373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.727978 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.727998 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.729687 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.729693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.729928 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.729966 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.730056 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.730241 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":26}
I20260809 22:58:43.731587 26546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.724172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:43451 (local address 127.24.31.195:33915)
0809 22:58:43.724892 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.724912 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.725046 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.725445 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.725449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.725478 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:43.725831 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.725842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.727316 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.727321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.728910 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.728922 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.730673 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.730680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.730863 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.730914 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.731012 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.731140 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":1,"thread_start_us":454,"threads_started":1}
I20260809 22:58:43.731644 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.724333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:33915 (local address 127.24.31.194:43451)
0809 22:58:43.724512 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.724517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.725030 (+ 513us) server_negotiation.cc:407] Connection header received
0809 22:58:43.725214 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.725220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.725307 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.725411 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:43.726011 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.727160 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.729078 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.730235 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.730435 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.730506 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.731089 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.731240 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:58:43.731777 26125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bca656291e44ecbbb839a3e292091a7" candidate_uuid: "2269eb7f873942cd980b7cd5189dbcfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bdd1885cb88449a95fafbda76820ab7" is_pre_election: true
I20260809 22:58:43.732033 26125 raft_consensus.cc:2454] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2269eb7f873942cd980b7cd5189dbcfa in term 0.
I20260809 22:58:43.732606 26357 leader_election.cc:302] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2269eb7f873942cd980b7cd5189dbcfa, 7bdd1885cb88449a95fafbda76820ab7; no voters:
I20260809 22:58:43.733011 26519 raft_consensus.cc:2781] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:58:43.733196 26519 raft_consensus.cc:482] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:58:43.733327 26519 raft_consensus.cc:3037] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:43.733676 26274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bca656291e44ecbbb839a3e292091a7" candidate_uuid: "2269eb7f873942cd980b7cd5189dbcfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db161bf35bab42f18ef772519f6fb774" is_pre_election: true
I20260809 22:58:43.734146 26274 raft_consensus.cc:2454] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2269eb7f873942cd980b7cd5189dbcfa in term 0.
I20260809 22:58:43.736227 26519 raft_consensus.cc:504] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.736775 26519 leader_election.cc:288] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [CANDIDATE]: Term 1 election: Requested vote from peers 7bdd1885cb88449a95fafbda76820ab7 (127.24.31.193:39635), db161bf35bab42f18ef772519f6fb774 (127.24.31.194:43451)
I20260809 22:58:43.737354 26274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bca656291e44ecbbb839a3e292091a7" candidate_uuid: "2269eb7f873942cd980b7cd5189dbcfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db161bf35bab42f18ef772519f6fb774"
I20260809 22:58:43.737412 26125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bca656291e44ecbbb839a3e292091a7" candidate_uuid: "2269eb7f873942cd980b7cd5189dbcfa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bdd1885cb88449a95fafbda76820ab7"
I20260809 22:58:43.737584 26274 raft_consensus.cc:3037] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:43.737612 26125 raft_consensus.cc:3037] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:43.740741 26125 raft_consensus.cc:2454] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2269eb7f873942cd980b7cd5189dbcfa in term 1.
I20260809 22:58:43.741127 26274 raft_consensus.cc:2454] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2269eb7f873942cd980b7cd5189dbcfa in term 1.
I20260809 22:58:43.741175 26357 leader_election.cc:302] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2269eb7f873942cd980b7cd5189dbcfa, 7bdd1885cb88449a95fafbda76820ab7; no voters:
I20260809 22:58:43.741671 26519 raft_consensus.cc:2781] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:43.742117 26519 raft_consensus.cc:686] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 1 LEADER]: Becoming Leader. State: Replica: 2269eb7f873942cd980b7cd5189dbcfa, State: Running, Role: LEADER
I20260809 22:58:43.742522 26519 consensus_queue.cc:237] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.746121 25946 catalog_manager.cc:5495] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa reported cstate change: term changed from 0 to 1, leader changed from <none> to 2269eb7f873942cd980b7cd5189dbcfa (127.24.31.195). New cstate: current_term: 1 leader_uuid: "2269eb7f873942cd980b7cd5189dbcfa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } health_report { overall_health: HEALTHY } } }
W20260809 22:58:43.774725 26493 tablet.cc:2369] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 22:58:43 dist-test-slave-vxj2 krb5kdc[25897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316321, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:43.823045 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.807809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.24.31.254:39269)
0809 22:58:43.808018 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.808026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.808672 (+ 646us) server_negotiation.cc:407] Connection header received
0809 22:58:43.808899 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.808905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.808979 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.809091 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:43.810321 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.811229 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.812601 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.813004 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.813129 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.816865 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:43.816870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:43.816873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:43.816903 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:43.819522 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:43.820441 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:43.820454 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:43.820457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:43.820577 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:43.821050 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:43.821053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:43.821055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:43.821462 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:43.821650 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.822294 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.822677 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":190,"server-negotiator.queue_time_us":51}
I20260809 22:58:43.823082 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.807834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39269 (local address 127.0.0.1:58362)
0809 22:58:43.808551 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.808574 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.808755 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.809126 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.809130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.809862 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:43.810153 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.810164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.811360 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.811366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.812463 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.812477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.813222 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.813229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.813381 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.814112 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:43.814134 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:43.816643 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:43.819670 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:43.819682 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:43.819690 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:43.820325 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:43.820729 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:43.820735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:43.820739 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:43.820924 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:43.821701 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:43.821711 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:43.821938 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.822140 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.822484 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":406,"threads_started":1}
I20260809 22:58:43.828626 25946 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:58362:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:58:43.829974 25946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:58:43.837519 26098 tablet_service.cc:1467] Processing CreateTablet for tablet 8e40ef1bc689450c89e5b3df8784e5fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57ac69a822dd46a8b76a6cb4310c1c53]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:58:43.837879 26252 tablet_service.cc:1467] Processing CreateTablet for tablet 8e40ef1bc689450c89e5b3df8784e5fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57ac69a822dd46a8b76a6cb4310c1c53]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:58:43.838182 26098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e40ef1bc689450c89e5b3df8784e5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:43.838327 26252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e40ef1bc689450c89e5b3df8784e5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:43.840445 26412 tablet_service.cc:1467] Processing CreateTablet for tablet 8e40ef1bc689450c89e5b3df8784e5fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57ac69a822dd46a8b76a6cb4310c1c53]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:58:43.840996 26412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e40ef1bc689450c89e5b3df8784e5fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:43.845808 26512 tablet_bootstrap.cc:492] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: Bootstrap starting.
I20260809 22:58:43.847221 26512 tablet_bootstrap.cc:654] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:43.848266 26514 tablet_bootstrap.cc:492] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: Bootstrap starting.
I20260809 22:58:43.849766 26514 tablet_bootstrap.cc:654] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:43.852216 26513 tablet_bootstrap.cc:492] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: Bootstrap starting.
I20260809 22:58:43.852382 26514 tablet_bootstrap.cc:492] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: No bootstrap required, opened a new log
I20260809 22:58:43.852560 26514 ts_tablet_manager.cc:1397] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 22:58:43.853338 26514 raft_consensus.cc:348] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.853581 26514 raft_consensus.cc:374] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:43.853698 26514 raft_consensus.cc:729] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db161bf35bab42f18ef772519f6fb774, State: Initialized, Role: FOLLOWER
I20260809 22:58:43.853955 26513 tablet_bootstrap.cc:654] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:43.854032 26514 consensus_queue.cc:260] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.854538 26523 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.854898 26523 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:43.855367 26514 ts_tablet_manager.cc:1428] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:43.865572 26513 tablet_bootstrap.cc:492] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: No bootstrap required, opened a new log
I20260809 22:58:43.865765 26513 ts_tablet_manager.cc:1397] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.001s
I20260809 22:58:43.865922 26512 tablet_bootstrap.cc:492] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: No bootstrap required, opened a new log
I20260809 22:58:43.866101 26512 ts_tablet_manager.cc:1397] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.011s
I20260809 22:58:43.866643 26513 raft_consensus.cc:348] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.866955 26513 raft_consensus.cc:374] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:43.867029 26513 raft_consensus.cc:729] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2269eb7f873942cd980b7cd5189dbcfa, State: Initialized, Role: FOLLOWER
I20260809 22:58:43.866916 26512 raft_consensus.cc:348] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.867168 26512 raft_consensus.cc:374] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:43.867246 26512 raft_consensus.cc:729] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bdd1885cb88449a95fafbda76820ab7, State: Initialized, Role: FOLLOWER
I20260809 22:58:43.867321 26513 consensus_queue.cc:260] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.867532 26512 consensus_queue.cc:260] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.867806 26519 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.868139 26519 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:43.868140 26512 ts_tablet_manager.cc:1428] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 22:58:43.868011 26518 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.868403 26518 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:43.869462 26513 ts_tablet_manager.cc:1428] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 22:58:43.921298 26523 raft_consensus.cc:482] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:43.921633 26523 raft_consensus.cc:504] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.922451 26523 leader_election.cc:288] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bdd1885cb88449a95fafbda76820ab7 (127.24.31.193:39635), 2269eb7f873942cd980b7cd5189dbcfa (127.24.31.195:37065)
I20260809 22:58:43.923071 26125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e40ef1bc689450c89e5b3df8784e5fb" candidate_uuid: "db161bf35bab42f18ef772519f6fb774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bdd1885cb88449a95fafbda76820ab7" is_pre_election: true
I20260809 22:58:43.923188 26441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e40ef1bc689450c89e5b3df8784e5fb" candidate_uuid: "db161bf35bab42f18ef772519f6fb774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2269eb7f873942cd980b7cd5189dbcfa" is_pre_election: true
I20260809 22:58:43.923380 26125 raft_consensus.cc:2454] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db161bf35bab42f18ef772519f6fb774 in term 0.
I20260809 22:58:43.923480 26441 raft_consensus.cc:2454] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db161bf35bab42f18ef772519f6fb774 in term 0.
I20260809 22:58:43.923897 26202 leader_election.cc:302] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bdd1885cb88449a95fafbda76820ab7, db161bf35bab42f18ef772519f6fb774; no voters:
I20260809 22:58:43.924275 26523 raft_consensus.cc:2781] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:58:43.924360 26523 raft_consensus.cc:482] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:58:43.924496 26523 raft_consensus.cc:3037] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:43.926903 26523 raft_consensus.cc:504] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.927502 26523 leader_election.cc:288] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [CANDIDATE]: Term 1 election: Requested vote from peers 7bdd1885cb88449a95fafbda76820ab7 (127.24.31.193:39635), 2269eb7f873942cd980b7cd5189dbcfa (127.24.31.195:37065)
I20260809 22:58:43.927868 26125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e40ef1bc689450c89e5b3df8784e5fb" candidate_uuid: "db161bf35bab42f18ef772519f6fb774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bdd1885cb88449a95fafbda76820ab7"
I20260809 22:58:43.928107 26125 raft_consensus.cc:3037] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:43.928087 26441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e40ef1bc689450c89e5b3df8784e5fb" candidate_uuid: "db161bf35bab42f18ef772519f6fb774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2269eb7f873942cd980b7cd5189dbcfa"
I20260809 22:58:43.928288 26441 raft_consensus.cc:3037] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:43.930450 26125 raft_consensus.cc:2454] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db161bf35bab42f18ef772519f6fb774 in term 1.
I20260809 22:58:43.930869 26202 leader_election.cc:302] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bdd1885cb88449a95fafbda76820ab7, db161bf35bab42f18ef772519f6fb774; no voters:
I20260809 22:58:43.931169 26523 raft_consensus.cc:2781] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:43.931380 26523 raft_consensus.cc:686] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [term 1 LEADER]: Becoming Leader. State: Replica: db161bf35bab42f18ef772519f6fb774, State: Running, Role: LEADER
I20260809 22:58:43.931792 26441 raft_consensus.cc:2454] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db161bf35bab42f18ef772519f6fb774 in term 1.
I20260809 22:58:43.931665 26523 consensus_queue.cc:237] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } }
I20260809 22:58:43.933501 26520 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: TxnStatusTablet state changed. Reason: New leader db161bf35bab42f18ef772519f6fb774. Latest consensus state: current_term: 1 leader_uuid: "db161bf35bab42f18ef772519f6fb774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.933809 26520 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:58:43.934923 25945 catalog_manager.cc:5495] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 reported cstate change: term changed from 0 to 1, leader changed from <none> to db161bf35bab42f18ef772519f6fb774 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "db161bf35bab42f18ef772519f6fb774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } health_report { overall_health: UNKNOWN } } }
I20260809 22:58:43.941716 26562 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:58:43.942090 26562 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:58:43.956182 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.950429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43764 (local address 127.24.31.194:43451)
0809 22:58:43.950614 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.950618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.950725 (+ 107us) server_negotiation.cc:407] Connection header received
0809 22:58:43.951034 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.951039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.951112 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.951198 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:43.951661 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.952478 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.954429 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.954877 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.954981 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.955529 (+ 548us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:43.955625 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:43.955695 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.955862 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.955920 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:58:43.956466 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.950332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:43451 (local address 127.0.0.1:43764)
0809 22:58:43.950621 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:43.950645 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:43.950814 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:43.951273 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:43.951279 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:43.951293 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:58:43.951511 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.951524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.952628 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.952634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.954283 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:43.954300 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.955043 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.955050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:43.955274 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.955347 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:58:43.955724 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:58:43.955745 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 22:58:43.955839 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 22:58:43.955999 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 22:58:43.960664 26441 raft_consensus.cc:1261] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa [term 1 FOLLOWER]: Refusing update from remote peer db161bf35bab42f18ef772519f6fb774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:43.960777 26125 raft_consensus.cc:1261] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Refusing update from remote peer db161bf35bab42f18ef772519f6fb774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:43.961320 26520 consensus_queue.cc:1035] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:43.961658 26523 consensus_queue.cc:1035] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:43.964682 26518 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: TxnStatusTablet state changed. Reason: New leader db161bf35bab42f18ef772519f6fb774. Latest consensus state: current_term: 1 leader_uuid: "db161bf35bab42f18ef772519f6fb774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.965103 26518 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:43.973712 26528 mvcc.cc:205] Tried to move back new op lower bound from 7316751662936915968 to 7316751662827040768. Current Snapshot: MvccSnapshot[applied={T|T < 7316751662936915968}]
I20260809 22:58:43.975268 26518 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db161bf35bab42f18ef772519f6fb774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.975279 26519 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: TxnStatusTablet state changed. Reason: New leader db161bf35bab42f18ef772519f6fb774. Latest consensus state: current_term: 1 leader_uuid: "db161bf35bab42f18ef772519f6fb774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.975514 26518 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P 7bdd1885cb88449a95fafbda76820ab7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:43.975581 26519 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:43.975493 26520 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db161bf35bab42f18ef772519f6fb774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.975800 26520 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:58:43.978641 26534 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "db161bf35bab42f18ef772519f6fb774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.979048 26534 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P db161bf35bab42f18ef772519f6fb774: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:58:43.981853 26519 tablet_replica.cc:439] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "db161bf35bab42f18ef772519f6fb774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 } } peers { permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 } } peers { permanent_uuid: "2269eb7f873942cd980b7cd5189dbcfa" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 37065 } } }
I20260809 22:58:43.982164 26519 tablet_replica.cc:442] T 8e40ef1bc689450c89e5b3df8784e5fb P 2269eb7f873942cd980b7cd5189dbcfa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:43.992044 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:43.984728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.24.31.195:37065)
0809 22:58:43.985672 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:43.985680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:43.985718 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:58:43.985815 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:43.985820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:43.985918 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:43.986097 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:43.986610 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.987853 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.989371 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:43.990022 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:43.990156 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:43.990577 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:43.991385 (+ 808us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:43.991558 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:43.991609 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 22:58:43.991697 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":622,"threads_started":1}
I20260809 22:58:44.008968 26572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:44.002692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:43451 (local address 127.24.31.195:57235)
0809 22:58:44.003359 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:44.003374 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:44.003496 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:44.003821 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:44.003824 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:44.003833 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:58:44.004069 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:44.004076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.005560 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.005564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:44.006867 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:44.006880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.007843 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.007847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:44.007975 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:44.008026 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:58:44.008476 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:58:44.008485 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:58:44.008558 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 22:58:44.008661 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":379,"threads_started":1}
I20260809 22:58:44.009028 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:44.002718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:57235 (local address 127.24.31.194:43451)
0809 22:58:44.002934 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:44.002940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:44.003467 (+ 527us) server_negotiation.cc:407] Connection header received
0809 22:58:44.003616 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:44.003622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:44.003704 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:44.003815 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:44.004212 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.005420 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.007025 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.007676 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.007810 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:44.008192 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:44.008385 (+ 193us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:44.008478 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:44.008604 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 22:58:44.008707 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:58:44.020196 26274 raft_consensus.cc:1261] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [term 1 FOLLOWER]: Refusing update from remote peer 2269eb7f873942cd980b7cd5189dbcfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:44.020776 26125 raft_consensus.cc:1261] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Refusing update from remote peer 2269eb7f873942cd980b7cd5189dbcfa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:44.021126 26547 consensus_queue.cc:1035] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "db161bf35bab42f18ef772519f6fb774" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:44.021636 26519 consensus_queue.cc:1035] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bdd1885cb88449a95fafbda76820ab7" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:44.060369 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:44.054832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:57607 (local address 127.24.31.195:37065)
0809 22:58:44.055018 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:44.055024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:44.055121 (+ 97us) server_negotiation.cc:407] Connection header received
0809 22:58:44.055291 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:44.055296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:44.055370 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:44.055473 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:44.056043 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.057085 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.058606 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.059029 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.059179 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:44.059627 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:44.059733 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:44.059831 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:44.059977 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 22:58:44.060057 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:58:44.060530 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:44.054405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:37065 (local address 127.24.31.194:57607)
0809 22:58:44.055032 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:44.055054 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:44.055177 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:44.055478 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:44.055480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:44.055488 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:58:44.055819 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:44.055827 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.057234 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.057240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:44.058458 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:44.058471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.059223 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.059232 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:44.059375 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:44.059436 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:58:44.059834 (+ 398us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:58:44.059848 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 22:58:44.059952 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 22:58:44.060109 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":496,"threads_started":1}
I20260809 22:58:44.100952 25945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:44.101289 25945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:44.105000 25945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40dc3485d8d04cd1ab75b32e0e71c088: Sending DeleteTablet for 3 replicas of tablet 5bca656291e44ecbbb839a3e292091a7
I20260809 22:58:44.106113 26098 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bca656291e44ecbbb839a3e292091a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:44 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:49114
I20260809 22:58:44.106177 26252 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bca656291e44ecbbb839a3e292091a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:44 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:43744
I20260809 22:58:44.106904 26412 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bca656291e44ecbbb839a3e292091a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:44 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:50330
I20260809 22:58:44.116855 26583 tablet_replica.cc:330] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: stopping tablet replica
I20260809 22:58:44.117313 26583 raft_consensus.cc:2229] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:44.117738 26583 raft_consensus.cc:2258] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:44.119071 26583 ts_tablet_manager.cc:1905] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:44.119656 26582 tablet_replica.cc:330] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: stopping tablet replica
I20260809 22:58:44.119910 26582 raft_consensus.cc:2229] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:44.127983 26583 ts_tablet_manager.cc:1918] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:58:44.128345 26582 raft_consensus.cc:2258] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:44.129547 26582 ts_tablet_manager.cc:1905] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:44.136850 26582 ts_tablet_manager.cc:1918] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:58:44.137081 26583 log.cc:1196] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/5bca656291e44ecbbb839a3e292091a7
I20260809 22:58:44.137609 26584 tablet_replica.cc:330] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: stopping tablet replica
I20260809 22:58:44.137821 26582 log.cc:1196] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/5bca656291e44ecbbb839a3e292091a7
I20260809 22:58:44.137862 26584 raft_consensus.cc:2229] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:58:44.138283 26582 ts_tablet_manager.cc:1939] T 5bca656291e44ecbbb839a3e292091a7 P 7bdd1885cb88449a95fafbda76820ab7: Deleting consensus metadata
I20260809 22:58:44.138566 26584 raft_consensus.cc:2258] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:44.139468 25932 catalog_manager.cc:4841] TS 7bdd1885cb88449a95fafbda76820ab7 (127.24.31.193:39635): tablet 5bca656291e44ecbbb839a3e292091a7 (table test-table [id=a3c7da70759c46a4bf5052035503ad38]) successfully deleted
I20260809 22:58:44.139878 26583 ts_tablet_manager.cc:1939] T 5bca656291e44ecbbb839a3e292091a7 P db161bf35bab42f18ef772519f6fb774: Deleting consensus metadata
I20260809 22:58:44.140668 26584 ts_tablet_manager.cc:1905] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:44.141000 25932 catalog_manager.cc:4841] TS db161bf35bab42f18ef772519f6fb774 (127.24.31.194:43451): tablet 5bca656291e44ecbbb839a3e292091a7 (table test-table [id=a3c7da70759c46a4bf5052035503ad38]) successfully deleted
I20260809 22:58:44.148533 26584 ts_tablet_manager.cc:1918] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:58:44.148674 26584 log.cc:1196] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/5bca656291e44ecbbb839a3e292091a7
I20260809 22:58:44.149217 26584 ts_tablet_manager.cc:1939] T 5bca656291e44ecbbb839a3e292091a7 P 2269eb7f873942cd980b7cd5189dbcfa: Deleting consensus metadata
I20260809 22:58:44.150353 25931 catalog_manager.cc:4841] TS 2269eb7f873942cd980b7cd5189dbcfa (127.24.31.195:37065): tablet 5bca656291e44ecbbb839a3e292091a7 (table test-table [id=a3c7da70759c46a4bf5052035503ad38]) successfully deleted
Aug 09 22:58:44 dist-test-slave-vxj2 krb5kdc[25897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316323, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:58:44.163683 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:44.135450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.24.31.193:39635)
0809 22:58:44.135642 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:44.135649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:44.139639 (+ 3990us) server_negotiation.cc:407] Connection header received
0809 22:58:44.141803 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:44.141809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:44.141911 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:44.141983 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:44.144112 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.145309 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.147259 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.147939 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.148039 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:44.153733 (+ 5694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:44.153751 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:44.153771 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:44.153825 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:44.159735 (+ 5910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.160809 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.160822 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.160829 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.161048 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.161598 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.161606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.161612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.162161 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:44.162511 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:44.163147 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 22:58:44.163416 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 22:58:44.165133 26171 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49126
I20260809 22:58:44.183374 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:44.168626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58370 (local address 127.24.31.254:39269)
0809 22:58:44.168859 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:44.168865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:44.169182 (+ 317us) server_negotiation.cc:407] Connection header received
0809 22:58:44.169586 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:44.169592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:44.169673 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:44.169793 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:44.171113 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.172359 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.174149 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.174588 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.174729 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:44.177261 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:44.177266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:44.177270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:44.177307 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:44.180605 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.181297 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.181303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.181306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.181451 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.181857 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.181861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.181865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.182197 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:44.182486 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:44.182836 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 22:58:44.183063 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 22:58:44.184165 25945 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58370
I20260809 22:58:44.186307 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26027
I20260809 22:58:44.205803 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:44.187775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49142 (local address 127.24.31.193:39635)
0809 22:58:44.187991 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:44.187997 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:44.193737 (+ 5740us) server_negotiation.cc:407] Connection header received
0809 22:58:44.193949 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:44.193954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:44.194040 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:44.194156 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:44.195374 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.196158 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.197497 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.197933 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.198077 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:44.200338 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:44.200346 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:44.200355 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:44.200387 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:44.203109 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.203769 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.203775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.203779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.203915 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.204297 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.204302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.204305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.204578 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:44.204866 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:44.205224 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 22:58:44.205458 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 22:58:44.206635 26171 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49142
W20260809 22:58:44.207257 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26027: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:44.207455 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 26027
I20260809 22:58:44.231035 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26180
Aug 09 22:58:44 dist-test-slave-vxj2 krb5kdc[25897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316323, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 22:58:44.247838 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:44.231882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43766 (local address 127.24.31.194:43451)
0809 22:58:44.232088 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:44.232094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:44.232164 (+ 70us) server_negotiation.cc:407] Connection header received
0809 22:58:44.232428 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:44.232434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:44.232524 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:44.232638 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:44.233976 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.234741 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.235987 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.236488 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.236643 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:44.241498 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:44.241513 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:44.241528 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:44.241566 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:44.244628 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.245440 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.245454 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.245456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.245597 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.246012 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.246018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.246020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.246436 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:44.246786 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:44.247112 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 22:58:44.247392 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 22:58:44.248548 26327 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43766
W20260809 22:58:44.249339 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26180: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:44.249539 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 26180
I20260809 22:58:44.278121 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26337
Aug 09 22:58:44 dist-test-slave-vxj2 krb5kdc[25897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316323, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 22:58:44.295001 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:44.278993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.24.31.195:37065)
0809 22:58:44.279202 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:44.279208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:44.279299 (+ 91us) server_negotiation.cc:407] Connection header received
0809 22:58:44.279511 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:44.279517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:44.279625 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:44.279745 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:44.281011 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.281928 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.283178 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:44.283652 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:44.283809 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:44.288535 (+ 4726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:44.288546 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:44.288561 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:44.288593 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:44.291960 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.292651 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.292666 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.292669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.292816 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:44.293341 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:44.293346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:44.293349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:44.293742 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:44.294179 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:44.294383 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 22:58:44.294601 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 22:58:44.295732 26485 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50350
W20260809 22:58:44.296415 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26337: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:44.296587 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 26337
I20260809 22:58:44.318220 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 25914
I20260809 22:58:44.319568 25991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:44.439976 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 25914
2026-08-09T22:58:44Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4677 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:44 dist-test-slave-vxj2 krb5kdc[26594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:44 dist-test-slave-vxj2 krb5kdc[26594](info): set up 2 sockets
Aug 09 22:58:44 dist-test-slave-vxj2 krb5kdc[26594](info): commencing operation
krb5kdc: starting...
W20260809 22:58:45.609730 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.006s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:58:45 dist-test-slave-vxj2 krb5kdc[26594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:58:45.811053 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39713
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:46.133183 26607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:46.133565 26607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:46.133749 26607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:46.140887 26607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:46.141036 26607 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:58:46.141137 26607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:46.141273 26607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:46.141367 26607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:46.141466 26607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:46.151250 26607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39713
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:46.152621 26607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:46.154011 26607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:46.166052 26612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:46.166092 26613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:46.166137 26615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:46.167655 26607 server_base.cc:1034] running on GCE node
I20260809 22:58:46.168447 26607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:46.169974 26607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:46.171193 26607 hybrid_clock.cc:648] HybridClock initialized: now 1786316326171137 us; error 27 us; skew 500 ppm
I20260809 22:58:46.171536 26607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:46 dist-test-slave-vxj2 krb5kdc[26594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:46.174705 26607 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:58:46.176149 26607 webserver.cc:460] Webserver started at http://127.24.31.254:33701/ using document root <none> and password file <none>
I20260809 22:58:46.176950 26607 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:46.177084 26607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:46.177438 26607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:46.180514 26607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "c94cad8671174defac7bd657b28c350c"
format_stamp: "Formatted at 2026-08-09 22:58:46 on dist-test-slave-vxj2"
server_key: "967106a1f4b873096f5c05068b7070c4"
server_key_iv: "37caf229bd04fc306b7dabefad2adbcb"
server_key_version: "encryptionkey@0"
I20260809 22:58:46.181391 26607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "c94cad8671174defac7bd657b28c350c"
format_stamp: "Formatted at 2026-08-09 22:58:46 on dist-test-slave-vxj2"
server_key: "967106a1f4b873096f5c05068b7070c4"
server_key_iv: "37caf229bd04fc306b7dabefad2adbcb"
server_key_version: "encryptionkey@0"
I20260809 22:58:46.186559 26607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 22:58:46.190255 26622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:46.191540 26607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:58:46.191742 26607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "c94cad8671174defac7bd657b28c350c"
format_stamp: "Formatted at 2026-08-09 22:58:46 on dist-test-slave-vxj2"
server_key: "967106a1f4b873096f5c05068b7070c4"
server_key_iv: "37caf229bd04fc306b7dabefad2adbcb"
server_key_version: "encryptionkey@0"
I20260809 22:58:46.191969 26607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:46.207348 26607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:46.211585 26607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:46.211876 26607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:46.233891 26607 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:39713
I20260809 22:58:46.233914 26693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:39713 every 8 connection(s)
I20260809 22:58:46.235316 26607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:46.239059 26694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:46.241348 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 26607
I20260809 22:58:46.241565 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:58:46.242362 24703 external_mini_cluster.cc:1428] Setting key bc5b2c8bde92592345762f2ca15a5aee
I20260809 22:58:46.252007 26694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c: Bootstrap starting.
I20260809 22:58:46.256033 26694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:46.257377 26694 log.cc:824] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:58:46 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:46.260983 26694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c: No bootstrap required, opened a new log
I20260809 22:58:46.267509 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:46.244516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.24.31.254:39713)
0809 22:58:46.245402 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:46.245413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:46.245455 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:58:46.247213 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:46.247235 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:46.247611 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:46.247952 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:46.249775 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.251097 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.252823 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.253563 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.253702 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:46.258928 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:46.258942 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:46.258962 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:46.259009 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:46.262935 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:46.263608 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:46.263616 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:46.263624 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:46.263834 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:46.264180 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:46.264184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:46.264188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:46.264951 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:46.265293 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:46.265653 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 22:58:46.266079 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":435,"threads_started":1}
I20260809 22:58:46.270803 26694 raft_consensus.cc:348] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94cad8671174defac7bd657b28c350c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39713 } }
I20260809 22:58:46.271129 26694 raft_consensus.cc:374] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:46.271206 26694 raft_consensus.cc:729] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c94cad8671174defac7bd657b28c350c, State: Initialized, Role: FOLLOWER
I20260809 22:58:46.271673 26694 consensus_queue.cc:260] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94cad8671174defac7bd657b28c350c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39713 } }
I20260809 22:58:46.271879 26694 raft_consensus.cc:388] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:46.271973 26694 raft_consensus.cc:482] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:46.272099 26694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:46.274277 26694 raft_consensus.cc:504] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94cad8671174defac7bd657b28c350c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39713 } }
I20260809 22:58:46.274791 26694 leader_election.cc:302] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c94cad8671174defac7bd657b28c350c; no voters:
I20260809 22:58:46.275341 26694 leader_election.cc:288] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:46.275472 26701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:46.276222 26701 raft_consensus.cc:686] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [term 1 LEADER]: Becoming Leader. State: Replica: c94cad8671174defac7bd657b28c350c, State: Running, Role: LEADER
I20260809 22:58:46.276751 26701 consensus_queue.cc:237] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94cad8671174defac7bd657b28c350c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39713 } }
I20260809 22:58:46.277591 26694 sys_catalog.cc:564] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:46.279724 26702 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c94cad8671174defac7bd657b28c350c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94cad8671174defac7bd657b28c350c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39713 } } }
I20260809 22:58:46.280167 26702 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:46.281749 26709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:46.281802 26703 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c94cad8671174defac7bd657b28c350c. Latest consensus state: current_term: 1 leader_uuid: "c94cad8671174defac7bd657b28c350c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c94cad8671174defac7bd657b28c350c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39713 } } }
I20260809 22:58:46.282121 26703 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:46.289139 26709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:46.299124 26709 catalog_manager.cc:1347] Generated new cluster ID: 03cd8cec544e402589862fb8ff45ee45
I20260809 22:58:46.299243 26709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:46.308368 26709 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:46.309737 26709 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:46.316491 26709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c: Generated new TSK 0
I20260809 22:58:46.317361 26709 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:58:46.374951 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39713
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:46.735467 26726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:46.735745 26726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:46.735891 26726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:46.743249 26726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:46.743366 26726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:46.743490 26726 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:58:46.743597 26726 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:58:46.743738 26726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:58:46.753728 26726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.31.254:39713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:46.755048 26726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:46.756417 26726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:46.769742 26734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:46.769701 26732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:46.769780 26731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:46.770553 26726 server_base.cc:1034] running on GCE node
I20260809 22:58:46.771206 26726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:46.772141 26726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:46.773393 26726 hybrid_clock.cc:648] HybridClock initialized: now 1786316326773335 us; error 54 us; skew 500 ppm
I20260809 22:58:46.773731 26726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:46 dist-test-slave-vxj2 krb5kdc[26594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:46.776975 26726 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:58:46.778753 26726 webserver.cc:460] Webserver started at http://127.24.31.193:45383/ using document root <none> and password file <none>
I20260809 22:58:46.779521 26726 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:46.779670 26726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:46.780041 26726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:46.783370 26726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "338b8db048e44dc29e1c35d4de03ae02"
format_stamp: "Formatted at 2026-08-09 22:58:46 on dist-test-slave-vxj2"
server_key: "ffb4c193056fe37bfbc11435f0f9aea8"
server_key_iv: "ad5a357d6ae3a5e3c0a1505128bbd474"
server_key_version: "encryptionkey@0"
I20260809 22:58:46.784201 26726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "338b8db048e44dc29e1c35d4de03ae02"
format_stamp: "Formatted at 2026-08-09 22:58:46 on dist-test-slave-vxj2"
server_key: "ffb4c193056fe37bfbc11435f0f9aea8"
server_key_iv: "ad5a357d6ae3a5e3c0a1505128bbd474"
server_key_version: "encryptionkey@0"
I20260809 22:58:46.789851 26726 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:58:46.793335 26743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:46.794672 26726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:58:46.794932 26726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "338b8db048e44dc29e1c35d4de03ae02"
format_stamp: "Formatted at 2026-08-09 22:58:46 on dist-test-slave-vxj2"
server_key: "ffb4c193056fe37bfbc11435f0f9aea8"
server_key_iv: "ad5a357d6ae3a5e3c0a1505128bbd474"
server_key_version: "encryptionkey@0"
I20260809 22:58:46.795248 26726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:46.838933 26726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:46.843135 26726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:46.843398 26726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:46.846167 26726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:46.846306 26726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:46.846486 26726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:46.846589 26726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:58:46 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:46.868693 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:46.846581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:38821 (local address 127.24.31.254:39713)
0809 22:58:46.847163 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:46.847170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:46.848372 (+ 1202us) server_negotiation.cc:407] Connection header received
0809 22:58:46.850532 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:46.850537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:46.850636 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:46.850790 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:46.853288 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.854393 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.857254 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.857678 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.857830 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:46.862017 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:46.862022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:46.862025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:46.862061 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:46.864654 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:46.865734 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:46.865739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:46.865742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:46.865918 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:46.866609 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:46.866614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:46.866618 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:46.866911 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:46.867097 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:46.868246 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0809 22:58:46.868422 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":267,"threads_started":1}
I20260809 22:58:46.870828 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:46.847299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39713 (local address 127.24.31.193:38821)
0809 22:58:46.848173 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:46.848230 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:46.850243 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:46.851091 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:46.851102 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:46.851954 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:46.853025 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:46.853054 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.854557 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.854567 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:46.857064 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:46.857078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.857956 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.857964 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:46.858229 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:46.858949 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:46.858972 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:46.861766 (+ 2794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:46.864845 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:46.864854 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:46.864871 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:46.865576 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:46.866061 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:46.866066 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:46.866069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:46.866436 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:46.867104 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:46.867114 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:46.867713 (+ 599us) client_negotiation.cc:769] Sending connection context
0809 22:58:46.868127 (+ 414us) client_negotiation.cc:240] Negotiation successful
0809 22:58:46.868870 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":364,"threads_started":1}
I20260809 22:58:46.896507 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:46.880439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:34809 (local address 127.24.31.254:39713)
0809 22:58:46.880666 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:46.880675 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:46.881369 (+ 694us) server_negotiation.cc:407] Connection header received
0809 22:58:46.881581 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:46.881589 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:46.881686 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:46.881799 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:46.883550 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.884260 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.886849 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.887437 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.887611 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:46.890480 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:46.890485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:46.890489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:46.890520 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:46.893070 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:46.894057 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:46.894060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:46.894063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:46.894194 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:46.894682 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:46.894690 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:46.894695 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:46.895116 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:46.895308 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:46.895845 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 22:58:46.896073 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:58:46.896631 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:46.880417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39713 (local address 127.24.31.193:34809)
0809 22:58:46.881235 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:46.881261 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:46.881436 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:46.881853 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:46.881859 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:46.882712 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:46.883365 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:46.883384 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.884435 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.884447 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:46.886615 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:46.886634 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:46.887676 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:46.887687 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:46.887990 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:46.888757 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:46.888781 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:46.890257 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:46.893238 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:46.893246 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:46.893253 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:46.893927 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:46.894321 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:46.894329 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:46.894333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:46.894524 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:46.895280 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:46.895287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:46.895552 (+ 265us) client_negotiation.cc:769] Sending connection context
0809 22:58:46.895729 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 22:58:46.896103 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":390,"threads_started":1}
I20260809 22:58:46.912448 26726 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:35851
I20260809 22:58:46.912523 26878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:35851 every 8 connection(s)
I20260809 22:58:46.913782 26726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:58:46.916009 26879 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39713
I20260809 22:58:46.916244 26879 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:46.916921 26879 heartbeater.cc:511] Master 127.24.31.254:39713 requested a full tablet report, sending...
I20260809 22:58:46.919051 26642 ts_manager.cc:194] Registered new tserver with Master: 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851)
I20260809 22:58:46.920348 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 26726
I20260809 22:58:46.920583 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:58:46.920524 26642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:38821
I20260809 22:58:46.921372 24703 external_mini_cluster.cc:1428] Setting key d59eebb92f45c951d1eb3e1fdad38482
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:58:46.964941 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39713
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:47.300016 26883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:47.300316 26883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:47.300526 26883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:47.308212 26883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:47.308328 26883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:47.308527 26883 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:58:47.308619 26883 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:58:47.308771 26883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:58:47.318519 26883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.31.254:39713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:47.319856 26883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:47.321242 26883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:47.334599 26889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:47.334568 26892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:47.334626 26890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:47.335263 26883 server_base.cc:1034] running on GCE node
I20260809 22:58:47.335891 26883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:47.336759 26883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:47.337994 26883 hybrid_clock.cc:648] HybridClock initialized: now 1786316327337920 us; error 60 us; skew 500 ppm
I20260809 22:58:47.338369 26883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:47 dist-test-slave-vxj2 krb5kdc[26594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316327, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:47.341607 26883 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:58:47.343225 26883 webserver.cc:460] Webserver started at http://127.24.31.194:44605/ using document root <none> and password file <none>
I20260809 22:58:47.343942 26883 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:47.344102 26883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:47.344439 26883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:47.347571 26883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "e1ef5c5ae3b348da95df3408381bccc0"
format_stamp: "Formatted at 2026-08-09 22:58:47 on dist-test-slave-vxj2"
server_key: "fb7276bd9ada74e9ab34d98b3e602e6d"
server_key_iv: "a7c82c29ccff78092a6a0babc378f736"
server_key_version: "encryptionkey@0"
I20260809 22:58:47.348389 26883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "e1ef5c5ae3b348da95df3408381bccc0"
format_stamp: "Formatted at 2026-08-09 22:58:47 on dist-test-slave-vxj2"
server_key: "fb7276bd9ada74e9ab34d98b3e602e6d"
server_key_iv: "a7c82c29ccff78092a6a0babc378f736"
server_key_version: "encryptionkey@0"
I20260809 22:58:47.353855 26883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:58:47.357510 26902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:47.358867 26883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:58:47.359052 26883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "e1ef5c5ae3b348da95df3408381bccc0"
format_stamp: "Formatted at 2026-08-09 22:58:47 on dist-test-slave-vxj2"
server_key: "fb7276bd9ada74e9ab34d98b3e602e6d"
server_key_iv: "a7c82c29ccff78092a6a0babc378f736"
server_key_version: "encryptionkey@0"
I20260809 22:58:47.359273 26883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:47.374173 26883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:47.378882 26883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:47.379285 26883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:47.383199 26883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:47.383347 26883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:47.383496 26883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:47.383610 26883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:58:47 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316327, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:47.408458 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:47.383549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:57887 (local address 127.24.31.254:39713)
0809 22:58:47.383739 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:47.383745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:47.385416 (+ 1671us) server_negotiation.cc:407] Connection header received
0809 22:58:47.388464 (+ 3048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:47.388472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:47.388572 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:47.388635 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:47.391125 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.392177 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.394761 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.395413 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.395581 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:47.399831 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:47.399843 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:47.399851 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:47.399945 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:47.404740 (+ 4795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:47.405602 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:47.405609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:47.405613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:47.405792 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:47.406566 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:47.406572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:47.406577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:47.406897 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:47.407080 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:47.407814 (+ 734us) server_negotiation.cc:299] Negotiation successful
0809 22:58:47.408050 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:58:47.409986 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:47.384205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39713 (local address 127.24.31.194:57887)
0809 22:58:47.385250 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:47.385300 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:47.388073 (+ 2773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:47.389111 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:47.389123 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:47.389975 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:47.390845 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:47.390881 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.392385 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.392390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:47.394579 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:47.394609 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.395677 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.395686 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:47.395935 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:47.396635 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:47.396687 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:47.399543 (+ 2856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:47.404910 (+ 5367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:47.404917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:47.404929 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:47.405455 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:47.405941 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:47.405950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:47.405958 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:47.406384 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:47.407053 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:47.407064 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:47.407423 (+ 359us) client_negotiation.cc:769] Sending connection context
0809 22:58:47.407696 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 22:58:47.408261 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":451,"threads_started":1}
I20260809 22:58:47.442075 26883 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:45661
I20260809 22:58:47.442137 27036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:45661 every 8 connection(s)
I20260809 22:58:47.443228 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:47.422041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:58855 (local address 127.24.31.254:39713)
0809 22:58:47.422303 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:47.422310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:47.426111 (+ 3801us) server_negotiation.cc:407] Connection header received
0809 22:58:47.426336 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:47.426345 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:47.426460 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:47.426612 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:47.428192 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.429668 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.431997 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.432791 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.432961 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:47.436202 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:47.436217 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:47.436222 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:47.436256 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:47.440217 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:47.440964 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:47.440969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:47.440971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:47.441202 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:47.441704 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:47.441709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:47.441712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:47.442053 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:47.442211 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:47.442648 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 22:58:47.442872 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 22:58:47.443338 26987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:47.422114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39713 (local address 127.24.31.194:58855)
0809 22:58:47.425959 (+ 3845us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:47.425983 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:47.426172 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:47.426668 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:47.426676 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:47.427475 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:47.427999 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:47.428020 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.429856 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.429866 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:47.431842 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:47.431855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.433043 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.433052 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:47.433294 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:47.434059 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:47.434100 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:47.435923 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:47.440362 (+ 4439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:47.440365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:47.440368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:47.440781 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:47.441336 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:47.441345 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:47.441352 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:47.441570 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:47.442196 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:47.442201 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:47.442387 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 22:58:47.442553 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 22:58:47.442895 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3685,"thread_start_us":3548,"threads_started":1}
I20260809 22:58:47.443548 26883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:58:47.445789 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 26883
I20260809 22:58:47.445802 27037 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39713
I20260809 22:58:47.445993 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:58:47.446066 27037 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:47.446583 24703 external_mini_cluster.cc:1428] Setting key d1585c97b0f05ec3811ef3a1144a0447
I20260809 22:58:47.446913 27037 heartbeater.cc:511] Master 127.24.31.254:39713 requested a full tablet report, sending...
I20260809 22:58:47.448628 26641 ts_manager.cc:194] Registered new tserver with Master: e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194:45661)
I20260809 22:58:47.449805 26641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:57887
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:58:47.491024 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39713
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:42727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:47.823558 27042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:47.823845 27042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:47.824057 27042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:47.831194 27042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:47.831327 27042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:47.831497 27042 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 22:58:47.831579 27042 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:58:47.831717 27042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:58:47.841202 27042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.31.254:39713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:47.842435 27042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:47.843710 27042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:47.856753 27048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:47.856818 27050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:47.856982 27052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:47.857682 27042 server_base.cc:1034] running on GCE node
I20260809 22:58:47.858259 27042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:47.859153 27042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:47.860386 27042 hybrid_clock.cc:648] HybridClock initialized: now 1786316327860335 us; error 62 us; skew 500 ppm
I20260809 22:58:47.860710 27042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:47 dist-test-slave-vxj2 krb5kdc[26594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316327, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:47.867452 27042 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:58:47.868960 27042 webserver.cc:460] Webserver started at http://127.24.31.195:35653/ using document root <none> and password file <none>
I20260809 22:58:47.869710 27042 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:47.869863 27042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:47.870172 27042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:47.873015 27042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "2869045cf32547be8b5fd9082e247011"
format_stamp: "Formatted at 2026-08-09 22:58:47 on dist-test-slave-vxj2"
server_key: "168b4ce40493fba3bd94c6b5c2b1a637"
server_key_iv: "2a1a94c519692e6b28a9706c6536495e"
server_key_version: "encryptionkey@0"
I20260809 22:58:47.873840 27042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "2869045cf32547be8b5fd9082e247011"
format_stamp: "Formatted at 2026-08-09 22:58:47 on dist-test-slave-vxj2"
server_key: "168b4ce40493fba3bd94c6b5c2b1a637"
server_key_iv: "2a1a94c519692e6b28a9706c6536495e"
server_key_version: "encryptionkey@0"
I20260809 22:58:47.879051 27042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:58:47.882557 27059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:47.883833 27042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:58:47.884012 27042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "2869045cf32547be8b5fd9082e247011"
format_stamp: "Formatted at 2026-08-09 22:58:47 on dist-test-slave-vxj2"
server_key: "168b4ce40493fba3bd94c6b5c2b1a637"
server_key_iv: "2a1a94c519692e6b28a9706c6536495e"
server_key_version: "encryptionkey@0"
I20260809 22:58:47.884217 27042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:47.901957 27042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:47.905889 27042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:47.906188 27042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:47.908897 27042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:47.909039 27042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:47.909210 27042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:47.909330 27042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 22:58:47.923478 26879 heartbeater.cc:503] Master 127.24.31.254:39713 was elected leader, sending a full tablet report...
Aug 09 22:58:47 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316327, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:47.935748 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:47.909289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:52271 (local address 127.24.31.254:39713)
0809 22:58:47.909489 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:47.909495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:47.911030 (+ 1535us) server_negotiation.cc:407] Connection header received
0809 22:58:47.914218 (+ 3188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:47.914226 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:47.914319 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:47.914400 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:47.917098 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.918113 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.921377 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.921945 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.922099 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:47.928315 (+ 6216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:47.928324 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:47.928331 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:47.928378 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:47.931397 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:47.932510 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:47.932514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:47.932516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:47.932634 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:47.933258 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:47.933266 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:47.933270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:47.933695 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:47.933860 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:47.934996 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0809 22:58:47.935297 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 22:58:47.937623 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:47.909910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39713 (local address 127.24.31.195:52271)
0809 22:58:47.910852 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:47.910904 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:47.913851 (+ 2947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:47.915038 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:47.915053 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:47.915900 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:47.916864 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:47.916896 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.918281 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.918288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:47.921213 (+ 2925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:47.921229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.922317 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.922327 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:47.922667 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:47.923999 (+ 1332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:47.924036 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:47.928040 (+ 4004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:47.931559 (+ 3519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:47.931573 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:47.931595 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:47.932377 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:47.932744 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:47.932751 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:47.932757 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:47.933084 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:47.933856 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:47.933867 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:47.934459 (+ 592us) client_negotiation.cc:769] Sending connection context
0809 22:58:47.934844 (+ 385us) client_negotiation.cc:240] Negotiation successful
0809 22:58:47.935575 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":350,"threads_started":1}
I20260809 22:58:47.963579 27042 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:36049
I20260809 22:58:47.964003 27191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:36049 every 8 connection(s)
I20260809 22:58:47.965061 27042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:58:47.967504 27192 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39713
I20260809 22:58:47.967732 27192 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:47.968382 27192 heartbeater.cc:511] Master 127.24.31.254:39713 requested a full tablet report, sending...
I20260809 22:58:47.969980 26641 ts_manager.cc:194] Registered new tserver with Master: 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:47.971022 26641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:52271
I20260809 22:58:47.971194 27186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:47.957405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39713 (local address 127.24.31.195:40005)
0809 22:58:47.958345 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:47.958374 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:47.958550 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:47.958909 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:47.958913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:47.959394 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:47.959687 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:47.959696 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.960720 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.960726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:47.961878 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:47.961890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.962976 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.962983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:47.963222 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:47.963995 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:47.964027 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:47.964942 (+ 915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:47.968542 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:47.968549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:47.968554 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:47.968955 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:47.969415 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:47.969419 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:47.969423 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:47.969606 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:47.970245 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:47.970249 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:47.970434 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 22:58:47.970577 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 22:58:47.970833 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"thread_start_us":631,"threads_started":1}
I20260809 22:58:47.971347 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:47.957537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:40005 (local address 127.24.31.254:39713)
0809 22:58:47.957811 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:47.957820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:47.958475 (+ 655us) server_negotiation.cc:407] Connection header received
0809 22:58:47.958680 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:47.958688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:47.958771 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:47.958879 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:47.959838 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.960571 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.962032 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:47.962749 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:47.962889 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:47.965145 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:47.965174 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:47.965181 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:47.965220 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:47.968408 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:47.969084 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:47.969092 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:47.969096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:47.969290 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:47.969714 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:47.969719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:47.969722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:47.970118 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:47.970264 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:47.970718 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 22:58:47.970939 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:58:47.971778 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 27042
I20260809 22:58:47.971967 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:58:47.972465 24703 external_mini_cluster.cc:1428] Setting key 3ca166ce2eb9d18997beec9fe89b8c1d
I20260809 22:58:47.975844 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:58:47 dist-test-slave-vxj2 krb5kdc[26594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:58:48 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:48.027081 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.013394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56378 (local address 127.24.31.254:39713)
0809 22:58:48.013685 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:48.013693 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:48.013900 (+ 207us) server_negotiation.cc:407] Connection header received
0809 22:58:48.014059 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:48.014064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:48.014135 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:48.014230 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:48.015584 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.016395 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.017641 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.018100 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.018201 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.022113 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:48.022118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:48.022121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:48.022154 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:48.024680 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:48.025393 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:48.025397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:48.025399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:48.025536 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:48.025886 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:48.025889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:48.025891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:48.026151 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:48.026325 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:48.026641 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 22:58:48.026812 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260809 22:58:48.031173 26641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:58:48.038246 26641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:58:48.064121 27210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.051111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:35851 (local address 127.0.0.1:48568)
0809 22:58:48.052755 (+ 1644us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:48.052776 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:48.052896 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:48.054359 (+ 1463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:48.054370 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:48.054423 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:48.054958 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.054973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.058412 (+ 3439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.058418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.061085 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.061100 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.062849 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.062856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.063092 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.063155 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 22:58:48.063339 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 22:58:48.063668 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1504,"thread_start_us":1807,"threads_started":1}
I20260809 22:58:48.064864 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.051451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48568 (local address 127.24.31.193:35851)
0809 22:58:48.053538 (+ 2087us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:48.053551 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:48.053587 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:58:48.053715 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:48.053727 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:48.054039 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:48.055187 (+ 1148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:48.055420 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.057332 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.061293 (+ 3961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.062615 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.062801 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.064248 (+ 1447us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:48.064332 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 22:58:48.064516 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1889,"thread_start_us":1746,"threads_started":1}
I20260809 22:58:48.066601 27209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.050498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:36049 (local address 127.0.0.1:40344)
0809 22:58:48.051824 (+ 1326us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:48.051873 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:48.052102 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:48.058673 (+ 6571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:48.058677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:48.058704 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:48.059199 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.059217 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.061509 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.061518 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.063833 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.063861 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.065801 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.065809 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.065993 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.066019 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 22:58:48.066098 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 22:58:48.066191 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":106,"thread_start_us":819,"threads_started":1}
I20260809 22:58:48.067072 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.053853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.24.31.194:45661)
0809 22:58:48.058283 (+ 4430us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:48.058290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:48.059134 (+ 844us) server_negotiation.cc:407] Connection header received
0809 22:58:48.059245 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:48.059253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:48.059503 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:48.059699 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:48.060514 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.061771 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.064286 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.065647 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.065774 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.065936 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:48.066539 (+ 603us) server_negotiation.cc:299] Negotiation successful
0809 22:58:48.066746 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4276,"thread_start_us":4139,"threads_started":1}
I20260809 22:58:48.067308 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.051112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40344 (local address 127.24.31.195:36049)
0809 22:58:48.058046 (+ 6934us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:48.058056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:48.058092 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:58:48.058194 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:48.058202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:48.058444 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:48.059440 (+ 996us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:48.059756 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.061370 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.064031 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.065559 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.065710 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.066728 (+ 1018us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:48.066813 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 22:58:48.066994 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6738,"thread_start_us":6540,"threads_started":1}
I20260809 22:58:48.067718 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.053356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:45661 (local address 127.0.0.1:34322)
0809 22:58:48.058787 (+ 5431us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:48.058816 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:48.058977 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:48.059922 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:48.059927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:48.059958 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:48.060318 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.060326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.061935 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.061941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.064120 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.064133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.066109 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.066118 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.066306 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.066337 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:58:48.066440 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 22:58:48.066603 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5283,"mutex_wait_us":1,"thread_start_us":5077,"threads_started":1}
I20260809 22:58:48.068979 26802 tablet_service.cc:1467] Processing CreateTablet for tablet 85613b60d91c41278e6e504068968df4 (DEFAULT_TABLE table=test-table [id=1c0b2196a4264703a418831e1ed8d115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:48.070559 26802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85613b60d91c41278e6e504068968df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:48.071043 26960 tablet_service.cc:1467] Processing CreateTablet for tablet 85613b60d91c41278e6e504068968df4 (DEFAULT_TABLE table=test-table [id=1c0b2196a4264703a418831e1ed8d115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:48.071074 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 85613b60d91c41278e6e504068968df4 (DEFAULT_TABLE table=test-table [id=1c0b2196a4264703a418831e1ed8d115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:48.072232 26960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85613b60d91c41278e6e504068968df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:48.072407 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85613b60d91c41278e6e504068968df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:48.084218 27215 tablet_bootstrap.cc:492] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: Bootstrap starting.
I20260809 22:58:48.086499 27216 tablet_bootstrap.cc:492] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: Bootstrap starting.
I20260809 22:58:48.088735 27217 tablet_bootstrap.cc:492] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: Bootstrap starting.
I20260809 22:58:48.088771 27215 tablet_bootstrap.cc:654] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:48.090296 27216 tablet_bootstrap.cc:654] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:48.090425 27215 log.cc:824] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:48.091555 27216 log.cc:824] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:48.091813 27217 tablet_bootstrap.cc:654] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:48.093039 27217 log.cc:824] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:48.093700 27215 tablet_bootstrap.cc:492] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: No bootstrap required, opened a new log
I20260809 22:58:48.094010 27215 ts_tablet_manager.cc:1397] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260809 22:58:48.094812 27216 tablet_bootstrap.cc:492] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: No bootstrap required, opened a new log
I20260809 22:58:48.095118 27216 ts_tablet_manager.cc:1397] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 22:58:48.096127 27217 tablet_bootstrap.cc:492] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: No bootstrap required, opened a new log
I20260809 22:58:48.096427 27217 ts_tablet_manager.cc:1397] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 22:58:48.103654 27217 raft_consensus.cc:348] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } }
I20260809 22:58:48.104048 27217 raft_consensus.cc:374] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:48.104153 27217 raft_consensus.cc:729] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2869045cf32547be8b5fd9082e247011, State: Initialized, Role: FOLLOWER
I20260809 22:58:48.104792 27217 consensus_queue.cc:260] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } }
I20260809 22:58:48.106150 27215 raft_consensus.cc:348] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } }
I20260809 22:58:48.106631 27215 raft_consensus.cc:374] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:48.106741 27215 raft_consensus.cc:729] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ef5c5ae3b348da95df3408381bccc0, State: Initialized, Role: FOLLOWER
I20260809 22:58:48.107154 27216 raft_consensus.cc:348] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } }
I20260809 22:58:48.107609 27216 raft_consensus.cc:374] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:48.107513 27215 consensus_queue.cc:260] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } }
I20260809 22:58:48.107717 27216 raft_consensus.cc:729] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 338b8db048e44dc29e1c35d4de03ae02, State: Initialized, Role: FOLLOWER
I20260809 22:58:48.108445 27216 consensus_queue.cc:260] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } }
I20260809 22:58:48.111027 27216 ts_tablet_manager.cc:1428] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: Time spent starting tablet: real 0.016s user 0.009s sys 0.008s
I20260809 22:58:48.112048 27037 heartbeater.cc:503] Master 127.24.31.254:39713 was elected leader, sending a full tablet report...
I20260809 22:58:48.112785 27217 ts_tablet_manager.cc:1428] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
I20260809 22:58:48.113121 27192 heartbeater.cc:503] Master 127.24.31.254:39713 was elected leader, sending a full tablet report...
I20260809 22:58:48.114002 27215 ts_tablet_manager.cc:1428] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: Time spent starting tablet: real 0.020s user 0.016s sys 0.003s
W20260809 22:58:48.166276 26880 tablet.cc:2369] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:58:48.195772 27039 tablet.cc:2369] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:58:48.217499 27193 tablet.cc:2369] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:58:48.415315 27225 raft_consensus.cc:482] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:48.415652 27225 raft_consensus.cc:504] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } }
I20260809 22:58:48.417543 27225 leader_election.cc:288] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049), 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851)
I20260809 22:58:48.424980 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.418100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:48735 (local address 127.24.31.195:36049)
0809 22:58:48.418312 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:48.418319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:48.419096 (+ 777us) server_negotiation.cc:407] Connection header received
0809 22:58:48.419231 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:48.419237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:48.419329 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:48.419463 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:48.420376 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.421436 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.423240 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.424024 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.424208 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.424288 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:48.424571 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 22:58:48.424698 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 22:58:48.425333 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.417933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:35025 (local address 127.24.31.193:35851)
0809 22:58:48.418139 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:48.418147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:48.419009 (+ 862us) server_negotiation.cc:407] Connection header received
0809 22:58:48.419233 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:48.419238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:48.419332 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:48.419444 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:48.420398 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.421466 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.423408 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.424249 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.424461 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.424553 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:48.424877 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 22:58:48.425023 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:58:48.425426 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.418013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:36049 (local address 127.24.31.194:48735)
0809 22:58:48.418967 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:48.418988 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:48.419093 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:48.419473 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:48.419478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:48.419571 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:48.420122 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.420142 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.421572 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.421578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.423078 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.423090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.424255 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.424261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.424403 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.424432 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 22:58:48.424546 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 22:58:48.424679 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":48,"spinlock_wait_cycles":21760,"thread_start_us":593,"threads_started":1}
I20260809 22:58:48.425490 27228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.417829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:35851 (local address 127.24.31.194:35025)
0809 22:58:48.418866 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:48.418889 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:48.419069 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:48.419473 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:48.419478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:48.419506 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:48.420229 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.420251 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.421615 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.421621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.423232 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.423247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.424525 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.424533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.424703 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.424727 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:58:48.424849 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 22:58:48.424988 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":132,"thread_start_us":636,"threads_started":1}
I20260809 22:58:48.426445 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85613b60d91c41278e6e504068968df4" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "338b8db048e44dc29e1c35d4de03ae02" is_pre_election: true
I20260809 22:58:48.426505 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85613b60d91c41278e6e504068968df4" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011" is_pre_election: true
I20260809 22:58:48.426947 26827 raft_consensus.cc:2454] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 0.
I20260809 22:58:48.426975 27143 raft_consensus.cc:2454] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 0.
I20260809 22:58:48.427615 26906 leader_election.cc:302] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2869045cf32547be8b5fd9082e247011, 338b8db048e44dc29e1c35d4de03ae02, e1ef5c5ae3b348da95df3408381bccc0; no voters:
I20260809 22:58:48.428053 27225 raft_consensus.cc:2781] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:58:48.428215 27225 raft_consensus.cc:482] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:58:48.428331 27225 raft_consensus.cc:3037] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:48.430752 27225 raft_consensus.cc:504] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } }
I20260809 22:58:48.431457 27225 leader_election.cc:288] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 election: Requested vote from peers 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049), 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851)
I20260809 22:58:48.431892 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85613b60d91c41278e6e504068968df4" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011"
I20260809 22:58:48.431929 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85613b60d91c41278e6e504068968df4" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "338b8db048e44dc29e1c35d4de03ae02"
I20260809 22:58:48.432152 26827 raft_consensus.cc:3037] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:48.432140 27143 raft_consensus.cc:3037] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:48.435681 27143 raft_consensus.cc:2454] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 1.
I20260809 22:58:48.435678 26827 raft_consensus.cc:2454] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 1.
I20260809 22:58:48.436118 26903 leader_election.cc:302] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 338b8db048e44dc29e1c35d4de03ae02, e1ef5c5ae3b348da95df3408381bccc0; no voters:
I20260809 22:58:48.436496 27225 raft_consensus.cc:2781] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:48.436954 27225 raft_consensus.cc:686] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 LEADER]: Becoming Leader. State: Replica: e1ef5c5ae3b348da95df3408381bccc0, State: Running, Role: LEADER
I20260809 22:58:48.437477 27225 consensus_queue.cc:237] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } }
I20260809 22:58:48.442415 26642 catalog_manager.cc:5495] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } health_report { overall_health: HEALTHY } } }
Aug 09 22:58:48 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:48.552958 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.539573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39713 (local address 127.0.0.1:56386)
0809 22:58:48.540447 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:48.540473 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:48.540643 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:48.541005 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:48.541009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:48.541813 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:48.542068 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.542078 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.543202 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.543209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.544161 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:48.544172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.544907 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.544915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:48.545099 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.545750 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:48.545767 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:48.548022 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:48.550771 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:48.550782 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:48.550787 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:48.551194 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:48.551518 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:48.551520 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:48.551523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:48.551659 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:48.552108 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:48.552115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:48.552262 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 22:58:48.552396 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 22:58:48.552594 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":401,"threads_started":1}
I20260809 22:58:48.553138 27238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:48.539573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56386 (local address 127.24.31.254:39713)
0809 22:58:48.540463 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:48.540471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:48.540551 (+ 80us) server_negotiation.cc:407] Connection header received
0809 22:58:48.540789 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:48.540801 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:48.540873 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:48.540983 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:48.542239 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.543067 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.544303 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:48.544713 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:48.544865 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:48.548220 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:48.548225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:48.548228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:48.548257 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:48.550651 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:48.551294 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:48.551298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:48.551300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:48.551424 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:48.551752 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:48.551755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:48.551757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:48.552001 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:48.552107 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:48.552489 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 22:58:48.552738 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":739,"thread_start_us":419,"threads_started":1}
I20260809 22:58:48.557837 26642 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:56386:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:58:48.558840 26642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:58:48.565580 26802 tablet_service.cc:1467] Processing CreateTablet for tablet 8d6792c679bd44ed9c60f0daa759bceb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4b435506c4ea4d65a99684c20548b44e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:58:48.565836 26960 tablet_service.cc:1467] Processing CreateTablet for tablet 8d6792c679bd44ed9c60f0daa759bceb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4b435506c4ea4d65a99684c20548b44e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:58:48.566663 26802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d6792c679bd44ed9c60f0daa759bceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:48.566660 26960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d6792c679bd44ed9c60f0daa759bceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:48.566316 27113 tablet_service.cc:1467] Processing CreateTablet for tablet 8d6792c679bd44ed9c60f0daa759bceb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4b435506c4ea4d65a99684c20548b44e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:58:48.567214 27113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d6792c679bd44ed9c60f0daa759bceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:48.573184 27217 tablet_bootstrap.cc:492] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: Bootstrap starting.
I20260809 22:58:48.573999 27216 tablet_bootstrap.cc:492] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: Bootstrap starting.
I20260809 22:58:48.574674 27215 tablet_bootstrap.cc:492] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: Bootstrap starting.
I20260809 22:58:48.575030 27217 tablet_bootstrap.cc:654] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:48.575695 27216 tablet_bootstrap.cc:654] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:48.576160 27215 tablet_bootstrap.cc:654] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:48.577510 27217 tablet_bootstrap.cc:492] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: No bootstrap required, opened a new log
I20260809 22:58:48.577663 27217 ts_tablet_manager.cc:1397] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 22:58:48.578406 27215 tablet_bootstrap.cc:492] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: No bootstrap required, opened a new log
I20260809 22:58:48.578572 27215 ts_tablet_manager.cc:1397] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 22:58:48.578485 27217 raft_consensus.cc:348] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:48.578747 27217 raft_consensus.cc:374] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:48.578855 27217 raft_consensus.cc:729] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2869045cf32547be8b5fd9082e247011, State: Initialized, Role: FOLLOWER
I20260809 22:58:48.579069 27216 tablet_bootstrap.cc:492] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: No bootstrap required, opened a new log
I20260809 22:58:48.579103 27217 consensus_queue.cc:260] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:48.579236 27216 ts_tablet_manager.cc:1397] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 22:58:48.579418 27215 raft_consensus.cc:348] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:48.579653 27215 raft_consensus.cc:374] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:48.579746 27215 raft_consensus.cc:729] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ef5c5ae3b348da95df3408381bccc0, State: Initialized, Role: FOLLOWER
I20260809 22:58:48.579790 27217 ts_tablet_manager.cc:1428] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:58:48.579604 27224 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:48.579943 27224 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:48.580030 27215 consensus_queue.cc:260] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:48.580039 27216 raft_consensus.cc:348] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:48.580265 27216 raft_consensus.cc:374] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:48.580358 27216 raft_consensus.cc:729] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 338b8db048e44dc29e1c35d4de03ae02, State: Initialized, Role: FOLLOWER
I20260809 22:58:48.580405 27225 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:48.580719 27225 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:48.580742 27215 ts_tablet_manager.cc:1428] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 22:58:48.580617 27216 consensus_queue.cc:260] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:48.581067 27226 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:48.581682 27226 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:48.582145 27216 ts_tablet_manager.cc:1428] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:58:48.905107 27225 consensus_queue.cc:1035] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:48.915165 27225 consensus_queue.cc:1035] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:48.950131 27246 raft_consensus.cc:482] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:48.950314 27246 raft_consensus.cc:504] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:48.950950 27246 leader_election.cc:288] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851), 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:48.951591 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6792c679bd44ed9c60f0daa759bceb" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "338b8db048e44dc29e1c35d4de03ae02" is_pre_election: true
I20260809 22:58:48.951591 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6792c679bd44ed9c60f0daa759bceb" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011" is_pre_election: true
I20260809 22:58:48.951905 26827 raft_consensus.cc:2454] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 0.
I20260809 22:58:48.951982 27143 raft_consensus.cc:2454] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 0.
I20260809 22:58:48.952382 26903 leader_election.cc:302] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 338b8db048e44dc29e1c35d4de03ae02, e1ef5c5ae3b348da95df3408381bccc0; no voters:
I20260809 22:58:48.952689 27246 raft_consensus.cc:2781] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:58:48.952798 27246 raft_consensus.cc:482] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:58:48.952865 27246 raft_consensus.cc:3037] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:48.954977 27246 raft_consensus.cc:504] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:48.955510 27246 leader_election.cc:288] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 election: Requested vote from peers 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851), 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:48.956027 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6792c679bd44ed9c60f0daa759bceb" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011"
I20260809 22:58:48.956027 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6792c679bd44ed9c60f0daa759bceb" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "338b8db048e44dc29e1c35d4de03ae02"
I20260809 22:58:48.956278 27143 raft_consensus.cc:3037] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:48.956296 26827 raft_consensus.cc:3037] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:48.959285 27143 raft_consensus.cc:2454] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 1.
I20260809 22:58:48.959291 26827 raft_consensus.cc:2454] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 1.
I20260809 22:58:48.959714 26903 leader_election.cc:302] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 338b8db048e44dc29e1c35d4de03ae02, e1ef5c5ae3b348da95df3408381bccc0; no voters:
I20260809 22:58:48.960026 27246 raft_consensus.cc:2781] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:48.960204 27246 raft_consensus.cc:686] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [term 1 LEADER]: Becoming Leader. State: Replica: e1ef5c5ae3b348da95df3408381bccc0, State: Running, Role: LEADER
I20260809 22:58:48.960475 27246 consensus_queue.cc:237] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:48.961905 27230 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: TxnStatusTablet state changed. Reason: New leader e1ef5c5ae3b348da95df3408381bccc0. Latest consensus state: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:48.962198 27230 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:58:48.963647 26642 catalog_manager.cc:5495] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260809 22:58:48.973639 27251 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:58:48.973989 27251 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:58:49.035861 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:49.029335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.24.31.194:45661)
0809 22:58:49.029966 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:49.029970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:49.029990 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:58:49.030066 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:49.030070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:49.030146 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:49.030278 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:49.030726 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.031557 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.033500 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.033970 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.034067 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:49.034622 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:49.035198 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:49.035310 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:49.035460 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 22:58:49.035531 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":348,"threads_started":1}
I20260809 22:58:49.035921 27239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:49.029144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:45661 (local address 127.0.0.1:34326)
0809 22:58:49.029458 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:49.029480 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:49.029619 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:49.030333 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:49.030338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:49.030352 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:58:49.030568 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:49.030579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.031686 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.031693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:49.033363 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:49.033379 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.034133 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.034139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:49.034354 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:49.034425 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:58:49.035305 (+ 880us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:58:49.035326 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 22:58:49.035419 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 22:58:49.035600 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 22:58:49.042883 26827 raft_consensus.cc:1261] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Refusing update from remote peer e1ef5c5ae3b348da95df3408381bccc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:49.043046 27143 raft_consensus.cc:1261] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Refusing update from remote peer e1ef5c5ae3b348da95df3408381bccc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:49.043608 27230 consensus_queue.cc:1035] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:49.043881 27246 consensus_queue.cc:1035] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:49.049836 27226 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: TxnStatusTablet state changed. Reason: New leader e1ef5c5ae3b348da95df3408381bccc0. Latest consensus state: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:49.050143 27226 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:49.051771 27224 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: TxnStatusTablet state changed. Reason: New leader e1ef5c5ae3b348da95df3408381bccc0. Latest consensus state: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:49.052047 27224 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:49.053063 27244 mvcc.cc:205] Tried to move back new op lower bound from 7316751683751616512 to 7316751683424096256. Current Snapshot: MvccSnapshot[applied={T|T < 7316751683751616512}]
I20260809 22:58:49.054800 27245 mvcc.cc:205] Tried to move back new op lower bound from 7316751683751616512 to 7316751683424096256. Current Snapshot: MvccSnapshot[applied={T|T < 7316751683751616512}]
I20260809 22:58:49.055905 27226 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:49.056137 27226 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P 338b8db048e44dc29e1c35d4de03ae02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:49.057937 27230 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:49.058218 27230 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:58:49.059058 27225 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:49.059392 27225 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P e1ef5c5ae3b348da95df3408381bccc0: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:58:49.060973 27224 tablet_replica.cc:439] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } } }
I20260809 22:58:49.061250 27224 tablet_replica.cc:442] T 8d6792c679bd44ed9c60f0daa759bceb P 2869045cf32547be8b5fd9082e247011: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:58:49.073272 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:49.067500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34334 (local address 127.24.31.194:45661)
0809 22:58:49.067676 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:49.067682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:49.068092 (+ 410us) server_negotiation.cc:407] Connection header received
0809 22:58:49.068244 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:49.068250 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:49.068341 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:49.068438 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:49.068849 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.070028 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.071329 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.072055 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.072182 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:49.072543 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:49.072664 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:49.072759 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:49.072858 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 22:58:49.072940 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:58:49.090338 27266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:49.084123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:45661 (local address 127.24.31.194:58559)
0809 22:58:49.084701 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:49.084717 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:49.084832 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:49.085217 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:49.085223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:49.085233 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:58:49.085473 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:49.085481 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.086915 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.086918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:49.088134 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:49.088146 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.089151 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.089183 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:49.089347 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:49.089399 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:58:49.089835 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:58:49.089850 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 22:58:49.089909 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 22:58:49.090011 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":32,"thread_start_us":276,"threads_started":1}
I20260809 22:58:49.090440 27254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:49.084214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:58559 (local address 127.24.31.194:45661)
0809 22:58:49.084390 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:49.084396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:49.084798 (+ 402us) server_negotiation.cc:407] Connection header received
0809 22:58:49.084961 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:49.084967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:49.085052 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:49.085180 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:49.085605 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.086790 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.088287 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:49.088984 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:49.089141 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:49.089614 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:49.089736 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:49.089829 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:49.089938 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 22:58:49.090013 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":66}
I20260809 22:58:49.156836 26641 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:49.157114 26641 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:49.160568 26641 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c: Sending DeleteTablet for 3 replicas of tablet 85613b60d91c41278e6e504068968df4
I20260809 22:58:49.161818 26802 tablet_service.cc:1514] Processing DeleteTablet for tablet 85613b60d91c41278e6e504068968df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:49 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:48568
I20260809 22:58:49.161924 27113 tablet_service.cc:1514] Processing DeleteTablet for tablet 85613b60d91c41278e6e504068968df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:49 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:40344
I20260809 22:58:49.161938 26960 tablet_service.cc:1514] Processing DeleteTablet for tablet 85613b60d91c41278e6e504068968df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:49 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:34322
I20260809 22:58:49.165966 26641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:58:49.167876 26641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:58:49.178467 27271 tablet_replica.cc:330] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: stopping tablet replica
I20260809 22:58:49.180533 26799 tablet_service.cc:1467] Processing CreateTablet for tablet 667085ace7ec4c2a976d00993b7d04ed (DEFAULT_TABLE table=test-table [id=2edc2d3a18e44c19bf07ebef730a0d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:49.181188 26799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667085ace7ec4c2a976d00993b7d04ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:49.181221 27271 raft_consensus.cc:2229] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:58:49.181567 26960 tablet_service.cc:1467] Processing CreateTablet for tablet 667085ace7ec4c2a976d00993b7d04ed (DEFAULT_TABLE table=test-table [id=2edc2d3a18e44c19bf07ebef730a0d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:49.181984 27271 raft_consensus.cc:2258] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:49.182085 26960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667085ace7ec4c2a976d00993b7d04ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:49.185683 27112 tablet_service.cc:1467] Processing CreateTablet for tablet 667085ace7ec4c2a976d00993b7d04ed (DEFAULT_TABLE table=test-table [id=2edc2d3a18e44c19bf07ebef730a0d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:49.186167 27112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667085ace7ec4c2a976d00993b7d04ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:49.187511 27271 ts_tablet_manager.cc:1905] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:49.188344 27269 tablet_replica.cc:330] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: stopping tablet replica
I20260809 22:58:49.188925 27269 raft_consensus.cc:2229] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:49.189483 27269 raft_consensus.cc:2258] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:49.192533 27269 ts_tablet_manager.cc:1905] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:49.195712 27270 tablet_replica.cc:330] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: stopping tablet replica
I20260809 22:58:49.196278 27272 tablet_bootstrap.cc:492] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: Bootstrap starting.
I20260809 22:58:49.196818 27270 raft_consensus.cc:2229] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:49.199329 27270 raft_consensus.cc:2258] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:49.199772 27272 tablet_bootstrap.cc:654] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:49.200734 27275 tablet_bootstrap.cc:492] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: Bootstrap starting.
I20260809 22:58:49.202672 27270 ts_tablet_manager.cc:1905] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:49.207001 27269 ts_tablet_manager.cc:1918] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:58:49.203145 27275 tablet_bootstrap.cc:654] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:49.207211 27269 log.cc:1196] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/85613b60d91c41278e6e504068968df4
I20260809 22:58:49.207798 27273 tablet_bootstrap.cc:492] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: Bootstrap starting.
I20260809 22:58:49.207880 27269 ts_tablet_manager.cc:1939] T 85613b60d91c41278e6e504068968df4 P 338b8db048e44dc29e1c35d4de03ae02: Deleting consensus metadata
I20260809 22:58:49.209368 27271 ts_tablet_manager.cc:1918] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:58:49.209545 27271 log.cc:1196] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/85613b60d91c41278e6e504068968df4
I20260809 22:58:49.209831 26623 catalog_manager.cc:4841] TS 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851): tablet 85613b60d91c41278e6e504068968df4 (table test-table [id=1c0b2196a4264703a418831e1ed8d115]) successfully deleted
I20260809 22:58:49.210126 27271 ts_tablet_manager.cc:1939] T 85613b60d91c41278e6e504068968df4 P e1ef5c5ae3b348da95df3408381bccc0: Deleting consensus metadata
I20260809 22:58:49.211817 27275 tablet_bootstrap.cc:492] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: No bootstrap required, opened a new log
I20260809 22:58:49.211972 27275 ts_tablet_manager.cc:1397] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260809 22:58:49.212213 26623 catalog_manager.cc:4841] TS e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194:45661): tablet 85613b60d91c41278e6e504068968df4 (table test-table [id=1c0b2196a4264703a418831e1ed8d115]) successfully deleted
I20260809 22:58:49.212845 27275 raft_consensus.cc:348] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:49.213104 27275 raft_consensus.cc:374] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:49.213277 27275 raft_consensus.cc:729] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2869045cf32547be8b5fd9082e247011, State: Initialized, Role: FOLLOWER
I20260809 22:58:49.213551 27275 consensus_queue.cc:260] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:49.213752 27273 tablet_bootstrap.cc:654] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:49.214887 27275 ts_tablet_manager.cc:1428] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:58:49.219317 27270 ts_tablet_manager.cc:1918] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:58:49.219522 27270 log.cc:1196] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/85613b60d91c41278e6e504068968df4
I20260809 22:58:49.220196 27270 ts_tablet_manager.cc:1939] T 85613b60d91c41278e6e504068968df4 P 2869045cf32547be8b5fd9082e247011: Deleting consensus metadata
I20260809 22:58:49.221308 27272 tablet_bootstrap.cc:492] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: No bootstrap required, opened a new log
I20260809 22:58:49.221487 27272 ts_tablet_manager.cc:1397] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.004s
I20260809 22:58:49.222105 26626 catalog_manager.cc:4841] TS 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049): tablet 85613b60d91c41278e6e504068968df4 (table test-table [id=1c0b2196a4264703a418831e1ed8d115]) successfully deleted
I20260809 22:58:49.222553 27273 tablet_bootstrap.cc:492] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: No bootstrap required, opened a new log
I20260809 22:58:49.222414 27272 raft_consensus.cc:348] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:49.222668 27272 raft_consensus.cc:374] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:49.222705 27273 ts_tablet_manager.cc:1397] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.007s
I20260809 22:58:49.222782 27272 raft_consensus.cc:729] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ef5c5ae3b348da95df3408381bccc0, State: Initialized, Role: FOLLOWER
I20260809 22:58:49.223037 27272 consensus_queue.cc:260] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:49.223445 27273 raft_consensus.cc:348] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:49.223657 27272 ts_tablet_manager.cc:1428] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:58:49.223685 27273 raft_consensus.cc:374] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:49.223783 27273 raft_consensus.cc:729] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 338b8db048e44dc29e1c35d4de03ae02, State: Initialized, Role: FOLLOWER
I20260809 22:58:49.224009 27273 consensus_queue.cc:260] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:49.224689 27273 ts_tablet_manager.cc:1428] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 22:58:49.331719 27268 raft_consensus.cc:482] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:49.332038 27268 raft_consensus.cc:504] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:49.333009 27268 leader_election.cc:288] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851), 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:49.333664 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667085ace7ec4c2a976d00993b7d04ed" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "338b8db048e44dc29e1c35d4de03ae02" is_pre_election: true
I20260809 22:58:49.333734 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667085ace7ec4c2a976d00993b7d04ed" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011" is_pre_election: true
I20260809 22:58:49.334008 27143 raft_consensus.cc:2454] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 0.
I20260809 22:58:49.334007 26827 raft_consensus.cc:2454] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 0.
I20260809 22:58:49.334542 26906 leader_election.cc:302] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2869045cf32547be8b5fd9082e247011, e1ef5c5ae3b348da95df3408381bccc0; no voters:
I20260809 22:58:49.334877 27268 raft_consensus.cc:2781] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:58:49.335022 27268 raft_consensus.cc:482] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:58:49.335139 27268 raft_consensus.cc:3037] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:49.338523 27268 raft_consensus.cc:504] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:49.339152 27268 leader_election.cc:288] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 election: Requested vote from peers 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851), 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:49.339704 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667085ace7ec4c2a976d00993b7d04ed" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "338b8db048e44dc29e1c35d4de03ae02"
I20260809 22:58:49.339716 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667085ace7ec4c2a976d00993b7d04ed" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011"
I20260809 22:58:49.339944 27143 raft_consensus.cc:3037] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:49.339988 26827 raft_consensus.cc:3037] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:49.342559 26827 raft_consensus.cc:2454] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 1.
I20260809 22:58:49.343106 26903 leader_election.cc:302] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 338b8db048e44dc29e1c35d4de03ae02, e1ef5c5ae3b348da95df3408381bccc0; no voters:
I20260809 22:58:49.343116 27143 raft_consensus.cc:2454] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 1.
I20260809 22:58:49.343463 27268 raft_consensus.cc:2781] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:49.343655 27268 raft_consensus.cc:686] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 1 LEADER]: Becoming Leader. State: Replica: e1ef5c5ae3b348da95df3408381bccc0, State: Running, Role: LEADER
I20260809 22:58:49.343981 27268 consensus_queue.cc:237] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:49.347185 26642 catalog_manager.cc:5495] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260809 22:58:49.413621 27143 raft_consensus.cc:1261] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Refusing update from remote peer e1ef5c5ae3b348da95df3408381bccc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:49.413961 26827 raft_consensus.cc:1261] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Refusing update from remote peer e1ef5c5ae3b348da95df3408381bccc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:49.414351 27230 consensus_queue.cc:1035] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:49.414994 27230 consensus_queue.cc:1035] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:49.422003 27249 mvcc.cc:205] Tried to move back new op lower bound from 7316751685271576576 to 7316751684995616768. Current Snapshot: MvccSnapshot[applied={T|T < 7316751685271576576}]
W20260809 22:58:49.446453 27244 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 22:58:51.049082 26642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:51.049360 26642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:51.053115 26642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c: Sending DeleteTablet for 3 replicas of tablet 667085ace7ec4c2a976d00993b7d04ed
I20260809 22:58:51.053911 26799 tablet_service.cc:1514] Processing DeleteTablet for tablet 667085ace7ec4c2a976d00993b7d04ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:51 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:48568
I20260809 22:58:51.054075 26960 tablet_service.cc:1514] Processing DeleteTablet for tablet 667085ace7ec4c2a976d00993b7d04ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:51 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:34322
I20260809 22:58:51.054086 27112 tablet_service.cc:1514] Processing DeleteTablet for tablet 667085ace7ec4c2a976d00993b7d04ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:51 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:40344
I20260809 22:58:51.071889 27306 tablet_replica.cc:330] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: stopping tablet replica
I20260809 22:58:51.072522 27308 tablet_replica.cc:330] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: stopping tablet replica
I20260809 22:58:51.072888 27307 tablet_replica.cc:330] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: stopping tablet replica
I20260809 22:58:51.072896 27306 raft_consensus.cc:2229] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:51.072891 27308 raft_consensus.cc:2229] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:51.073194 27306 raft_consensus.cc:2258] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:51.073185 27307 raft_consensus.cc:2229] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:58:51.073669 27307 raft_consensus.cc:2258] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:51.073741 27308 raft_consensus.cc:2258] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:51.074239 27306 ts_tablet_manager.cc:1905] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:51.074741 27307 ts_tablet_manager.cc:1905] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:51.075086 27308 ts_tablet_manager.cc:1905] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:51.081878 27306 ts_tablet_manager.cc:1918] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:58:51.082054 27306 log.cc:1196] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/667085ace7ec4c2a976d00993b7d04ed
I20260809 22:58:51.082062 27308 ts_tablet_manager.cc:1918] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:58:51.082305 27308 log.cc:1196] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/667085ace7ec4c2a976d00993b7d04ed
I20260809 22:58:51.082494 27307 ts_tablet_manager.cc:1918] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:58:51.082551 27306 ts_tablet_manager.cc:1939] T 667085ace7ec4c2a976d00993b7d04ed P 338b8db048e44dc29e1c35d4de03ae02: Deleting consensus metadata
I20260809 22:58:51.082652 27307 log.cc:1196] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/667085ace7ec4c2a976d00993b7d04ed
I20260809 22:58:51.082973 27308 ts_tablet_manager.cc:1939] T 667085ace7ec4c2a976d00993b7d04ed P 2869045cf32547be8b5fd9082e247011: Deleting consensus metadata
I20260809 22:58:51.083135 27307 ts_tablet_manager.cc:1939] T 667085ace7ec4c2a976d00993b7d04ed P e1ef5c5ae3b348da95df3408381bccc0: Deleting consensus metadata
I20260809 22:58:51.084019 26623 catalog_manager.cc:4841] TS 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851): tablet 667085ace7ec4c2a976d00993b7d04ed (table test-table [id=2edc2d3a18e44c19bf07ebef730a0d53]) successfully deleted
I20260809 22:58:51.084156 26626 catalog_manager.cc:4841] TS 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049): tablet 667085ace7ec4c2a976d00993b7d04ed (table test-table [id=2edc2d3a18e44c19bf07ebef730a0d53]) successfully deleted
I20260809 22:58:51.084362 26623 catalog_manager.cc:4841] TS e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194:45661): tablet 667085ace7ec4c2a976d00993b7d04ed (table test-table [id=2edc2d3a18e44c19bf07ebef730a0d53]) successfully deleted
Aug 09 22:58:51 dist-test-slave-vxj2 krb5kdc[26594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:58:51 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:51.120471 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.105536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56396 (local address 127.24.31.254:39713)
0809 22:58:51.106166 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:51.106171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:51.106212 (+ 41us) server_negotiation.cc:407] Connection header received
0809 22:58:51.106378 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:51.106382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:51.106457 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:51.106560 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:51.107938 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.108851 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.110193 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.110691 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.110822 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.114901 (+ 4079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:51.114907 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:51.114910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:51.114945 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:51.118042 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:51.118619 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:51.118622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:51.118624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:51.118755 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:51.119160 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:51.119166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:51.119170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:51.119510 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:51.119711 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:51.119967 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 22:58:51.120150 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":329,"threads_started":1}
I20260809 22:58:51.124311 26642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:58:51.124912 26642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:58:51.131827 26799 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb1e95ff3414340a50d094c71be2696 (DEFAULT_TABLE table=test-table [id=f561375c1bd242778737935a94c8ec4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:51.132053 26960 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb1e95ff3414340a50d094c71be2696 (DEFAULT_TABLE table=test-table [id=f561375c1bd242778737935a94c8ec4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:51.132335 26799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb1e95ff3414340a50d094c71be2696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:51.132241 27112 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb1e95ff3414340a50d094c71be2696 (DEFAULT_TABLE table=test-table [id=f561375c1bd242778737935a94c8ec4e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:51.132542 26960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb1e95ff3414340a50d094c71be2696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:51.132651 27112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb1e95ff3414340a50d094c71be2696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:51.138271 27318 tablet_bootstrap.cc:492] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: Bootstrap starting.
I20260809 22:58:51.139328 27319 tablet_bootstrap.cc:492] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: Bootstrap starting.
I20260809 22:58:51.140375 27318 tablet_bootstrap.cc:654] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:51.141194 27319 tablet_bootstrap.cc:654] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:51.143420 27319 tablet_bootstrap.cc:492] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: No bootstrap required, opened a new log
I20260809 22:58:51.143589 27319 ts_tablet_manager.cc:1397] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260809 22:58:51.143671 27320 tablet_bootstrap.cc:492] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: Bootstrap starting.
I20260809 22:58:51.144089 27318 tablet_bootstrap.cc:492] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: No bootstrap required, opened a new log
I20260809 22:58:51.144244 27318 ts_tablet_manager.cc:1397] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 22:58:51.144441 27319 raft_consensus.cc:348] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.144727 27319 raft_consensus.cc:374] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:51.144817 27319 raft_consensus.cc:729] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 338b8db048e44dc29e1c35d4de03ae02, State: Initialized, Role: FOLLOWER
I20260809 22:58:51.144999 27318 raft_consensus.cc:348] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.145056 27319 consensus_queue.cc:260] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.145272 27318 raft_consensus.cc:374] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:51.145371 27318 raft_consensus.cc:729] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ef5c5ae3b348da95df3408381bccc0, State: Initialized, Role: FOLLOWER
I20260809 22:58:51.145474 27320 tablet_bootstrap.cc:654] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:51.145628 27318 consensus_queue.cc:260] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.146725 27318 ts_tablet_manager.cc:1428] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:58:51.147547 27319 ts_tablet_manager.cc:1428] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 22:58:51.149235 27320 tablet_bootstrap.cc:492] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: No bootstrap required, opened a new log
I20260809 22:58:51.149411 27320 ts_tablet_manager.cc:1397] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 22:58:51.150223 27320 raft_consensus.cc:348] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.150485 27320 raft_consensus.cc:374] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:51.150642 27320 raft_consensus.cc:729] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2869045cf32547be8b5fd9082e247011, State: Initialized, Role: FOLLOWER
I20260809 22:58:51.150898 27320 consensus_queue.cc:260] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.151724 27320 ts_tablet_manager.cc:1428] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:58:51.303738 27301 raft_consensus.cc:482] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:51.304056 27301 raft_consensus.cc:504] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.304365 27323 raft_consensus.cc:482] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:51.304600 27323 raft_consensus.cc:504] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.304983 27301 leader_election.cc:288] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851), 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:51.305688 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb1e95ff3414340a50d094c71be2696" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011" is_pre_election: true
I20260809 22:58:51.305660 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb1e95ff3414340a50d094c71be2696" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "338b8db048e44dc29e1c35d4de03ae02" is_pre_election: true
I20260809 22:58:51.305960 26827 raft_consensus.cc:2454] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 0.
I20260809 22:58:51.305987 27143 raft_consensus.cc:2454] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 0.
I20260809 22:58:51.306449 26903 leader_election.cc:302] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 338b8db048e44dc29e1c35d4de03ae02, e1ef5c5ae3b348da95df3408381bccc0; no voters:
I20260809 22:58:51.306852 27301 raft_consensus.cc:2781] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:58:51.306978 27301 raft_consensus.cc:482] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:58:51.307088 27301 raft_consensus.cc:3037] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:51.311276 27301 raft_consensus.cc:504] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.312342 27323 leader_election.cc:288] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194:45661), 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:51.313292 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb1e95ff3414340a50d094c71be2696" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011"
I20260809 22:58:51.313607 27143 raft_consensus.cc:3037] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:51.318166 27143 raft_consensus.cc:2454] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 1.
I20260809 22:58:51.318249 26827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb1e95ff3414340a50d094c71be2696" candidate_uuid: "e1ef5c5ae3b348da95df3408381bccc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "338b8db048e44dc29e1c35d4de03ae02"
I20260809 22:58:51.318584 26827 raft_consensus.cc:3037] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:51.318624 26906 leader_election.cc:302] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2869045cf32547be8b5fd9082e247011, e1ef5c5ae3b348da95df3408381bccc0; no voters:
I20260809 22:58:51.318931 27301 leader_election.cc:288] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [CANDIDATE]: Term 1 election: Requested vote from peers 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851), 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:51.318996 27299 raft_consensus.cc:2781] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:51.319154 27299 raft_consensus.cc:686] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 LEADER]: Becoming Leader. State: Replica: e1ef5c5ae3b348da95df3408381bccc0, State: Running, Role: LEADER
I20260809 22:58:51.319494 27299 consensus_queue.cc:237] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.322064 26827 raft_consensus.cc:2454] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1ef5c5ae3b348da95df3408381bccc0 in term 1.
I20260809 22:58:51.324450 26641 catalog_manager.cc:5495] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "e1ef5c5ae3b348da95df3408381bccc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260809 22:58:51.327847 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.312494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:55147 (local address 127.24.31.194:45661)
0809 22:58:51.314998 (+ 2504us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:51.315007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:51.320501 (+ 5494us) server_negotiation.cc:407] Connection header received
0809 22:58:51.320573 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:51.320578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:51.320658 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:51.320749 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:51.321629 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.322987 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.325632 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.326793 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.326989 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.327077 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:51.327351 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 22:58:51.327479 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2347,"thread_start_us":4707,"threads_started":1}
I20260809 22:58:51.327894 27326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.306652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:45661 (local address 127.24.31.193:55147)
0809 22:58:51.317968 (+ 11316us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:51.317996 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:51.320360 (+ 2364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:51.321017 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:51.321022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:51.321048 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:51.321415 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:51.321426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.323156 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.323164 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:51.325432 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:51.325446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.326989 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.326995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:51.327175 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.327198 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:58:51.327312 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 22:58:51.327453 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11118,"mutex_wait_us":2,"thread_start_us":10901,"threads_started":1}
I20260809 22:58:51.328512 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.306854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:36049 (local address 127.24.31.193:42795)
0809 22:58:51.320695 (+ 13841us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:51.320722 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:51.320878 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:51.321654 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:51.321662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:51.321697 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:58:51.322357 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:51.322373 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.323874 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.323879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:51.325909 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:51.325925 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.327652 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.327659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:51.327862 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.327911 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 22:58:51.328019 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 22:58:51.328147 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13682,"thread_start_us":12503,"threads_started":1}
I20260809 22:58:51.328661 27328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.306881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:42795 (local address 127.24.31.195:36049)
0809 22:58:51.320461 (+ 13580us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:51.320467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:51.321030 (+ 563us) server_negotiation.cc:407] Connection header received
0809 22:58:51.321124 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:51.321132 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:51.321252 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:51.321364 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:58:51.322576 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.323694 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.326100 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.327210 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.327400 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.327472 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:51.328140 (+ 668us) server_negotiation.cc:299] Negotiation successful
0809 22:58:51.328291 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13426,"thread_start_us":13194,"threads_started":1}
I20260809 22:58:51.328800 26985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb1e95ff3414340a50d094c71be2696" candidate_uuid: "338b8db048e44dc29e1c35d4de03ae02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ef5c5ae3b348da95df3408381bccc0" is_pre_election: true
I20260809 22:58:51.329264 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb1e95ff3414340a50d094c71be2696" candidate_uuid: "338b8db048e44dc29e1c35d4de03ae02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011" is_pre_election: true
I20260809 22:58:51.329605 27143 raft_consensus.cc:2379] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 338b8db048e44dc29e1c35d4de03ae02 in current term 1: Already voted for candidate e1ef5c5ae3b348da95df3408381bccc0 in this term.
I20260809 22:58:51.330112 26747 leader_election.cc:302] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 338b8db048e44dc29e1c35d4de03ae02; no voters: 2869045cf32547be8b5fd9082e247011, e1ef5c5ae3b348da95df3408381bccc0
I20260809 22:58:51.330544 27323 raft_consensus.cc:2726] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 22:58:51.393953 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.389418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.24.31.194:45661)
0809 22:58:51.389595 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:51.389600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:51.389670 (+ 70us) server_negotiation.cc:407] Connection header received
0809 22:58:51.389857 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:51.389861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:51.389928 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:51.390007 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:51.390319 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.391054 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.392300 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.392680 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.392819 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.393260 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:51.393356 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:51.393425 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:51.393557 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 22:58:51.393618 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 22:58:51.403179 27143 raft_consensus.cc:1261] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Refusing update from remote peer e1ef5c5ae3b348da95df3408381bccc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:51.403337 26827 raft_consensus.cc:1261] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Refusing update from remote peer e1ef5c5ae3b348da95df3408381bccc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:51.403796 27301 consensus_queue.cc:1035] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:51.404290 27286 consensus_queue.cc:1035] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:51.447245 26641 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:51.447535 26641 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:51.450697 26641 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c: Sending DeleteTablet for 3 replicas of tablet 9eb1e95ff3414340a50d094c71be2696
I20260809 22:58:51.451442 26799 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb1e95ff3414340a50d094c71be2696 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:51 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:48568
I20260809 22:58:51.451745 27306 tablet_replica.cc:330] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: stopping tablet replica
I20260809 22:58:51.451749 27112 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb1e95ff3414340a50d094c71be2696 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:51 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:40344
I20260809 22:58:51.451763 26960 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb1e95ff3414340a50d094c71be2696 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:51 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:34322
I20260809 22:58:51.452008 27307 tablet_replica.cc:330] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: stopping tablet replica
I20260809 22:58:51.452016 27308 tablet_replica.cc:330] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: stopping tablet replica
I20260809 22:58:51.451983 27306 raft_consensus.cc:2229] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:51.452183 27307 raft_consensus.cc:2229] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:58:51.452219 27306 raft_consensus.cc:2258] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:51.452203 27308 raft_consensus.cc:2229] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:51.452410 27308 raft_consensus.cc:2258] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:51.452479 27307 raft_consensus.cc:2258] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:51.453621 27308 ts_tablet_manager.cc:1905] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:51.454015 27306 ts_tablet_manager.cc:1905] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:51.454056 27307 ts_tablet_manager.cc:1905] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:51.460470 27308 ts_tablet_manager.cc:1918] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:58:51.460470 27306 ts_tablet_manager.cc:1918] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:58:51.460635 27308 log.cc:1196] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/9eb1e95ff3414340a50d094c71be2696
I20260809 22:58:51.460646 27306 log.cc:1196] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/9eb1e95ff3414340a50d094c71be2696
I20260809 22:58:51.460722 27307 ts_tablet_manager.cc:1918] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:58:51.460875 27307 log.cc:1196] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/9eb1e95ff3414340a50d094c71be2696
I20260809 22:58:51.461072 27306 ts_tablet_manager.cc:1939] T 9eb1e95ff3414340a50d094c71be2696 P 338b8db048e44dc29e1c35d4de03ae02: Deleting consensus metadata
I20260809 22:58:51.461084 27308 ts_tablet_manager.cc:1939] T 9eb1e95ff3414340a50d094c71be2696 P 2869045cf32547be8b5fd9082e247011: Deleting consensus metadata
I20260809 22:58:51.461372 27307 ts_tablet_manager.cc:1939] T 9eb1e95ff3414340a50d094c71be2696 P e1ef5c5ae3b348da95df3408381bccc0: Deleting consensus metadata
I20260809 22:58:51.462237 26626 catalog_manager.cc:4841] TS 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049): tablet 9eb1e95ff3414340a50d094c71be2696 (table test-table [id=f561375c1bd242778737935a94c8ec4e]) successfully deleted
I20260809 22:58:51.462621 26623 catalog_manager.cc:4841] TS e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194:45661): tablet 9eb1e95ff3414340a50d094c71be2696 (table test-table [id=f561375c1bd242778737935a94c8ec4e]) successfully deleted
I20260809 22:58:51.462852 26623 catalog_manager.cc:4841] TS 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851): tablet 9eb1e95ff3414340a50d094c71be2696 (table test-table [id=f561375c1bd242778737935a94c8ec4e]) successfully deleted
I20260809 22:58:51.464015 26641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:58:51.464741 26641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:58:51.471767 26799 tablet_service.cc:1467] Processing CreateTablet for tablet ee2b6da779df44e389f533d4ebe6860f (DEFAULT_TABLE table=test-table [id=3976a5136deb4cfa9426e5182944486d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:51.472203 26799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee2b6da779df44e389f533d4ebe6860f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:51.472095 27112 tablet_service.cc:1467] Processing CreateTablet for tablet ee2b6da779df44e389f533d4ebe6860f (DEFAULT_TABLE table=test-table [id=3976a5136deb4cfa9426e5182944486d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:51.472569 27112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee2b6da779df44e389f533d4ebe6860f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:51.476817 27319 tablet_bootstrap.cc:492] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: Bootstrap starting.
I20260809 22:58:51.477631 26960 tablet_service.cc:1467] Processing CreateTablet for tablet ee2b6da779df44e389f533d4ebe6860f (DEFAULT_TABLE table=test-table [id=3976a5136deb4cfa9426e5182944486d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:58:51.478120 26960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee2b6da779df44e389f533d4ebe6860f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:51.478302 27319 tablet_bootstrap.cc:654] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:51.483916 27318 tablet_bootstrap.cc:492] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: Bootstrap starting.
I20260809 22:58:51.484263 27319 tablet_bootstrap.cc:492] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: No bootstrap required, opened a new log
I20260809 22:58:51.484486 27319 ts_tablet_manager.cc:1397] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260809 22:58:51.484860 27320 tablet_bootstrap.cc:492] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: Bootstrap starting.
I20260809 22:58:51.485574 27318 tablet_bootstrap.cc:654] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:51.485387 27319 raft_consensus.cc:348] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.485927 27319 raft_consensus.cc:374] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:51.486068 27319 raft_consensus.cc:729] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 338b8db048e44dc29e1c35d4de03ae02, State: Initialized, Role: FOLLOWER
I20260809 22:58:51.486431 27319 consensus_queue.cc:260] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.486728 27320 tablet_bootstrap.cc:654] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:51.487179 27319 ts_tablet_manager.cc:1428] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:58:51.490357 27318 tablet_bootstrap.cc:492] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: No bootstrap required, opened a new log
I20260809 22:58:51.490525 27318 ts_tablet_manager.cc:1397] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260809 22:58:51.491312 27318 raft_consensus.cc:348] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.491582 27318 raft_consensus.cc:374] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:51.491670 27318 raft_consensus.cc:729] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1ef5c5ae3b348da95df3408381bccc0, State: Initialized, Role: FOLLOWER
I20260809 22:58:51.491915 27318 consensus_queue.cc:260] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.492409 27318 ts_tablet_manager.cc:1428] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 22:58:51.493481 27320 tablet_bootstrap.cc:492] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: No bootstrap required, opened a new log
I20260809 22:58:51.493628 27320 ts_tablet_manager.cc:1397] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.003s
I20260809 22:58:51.494424 27320 raft_consensus.cc:348] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.494701 27320 raft_consensus.cc:374] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:51.494798 27320 raft_consensus.cc:729] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2869045cf32547be8b5fd9082e247011, State: Initialized, Role: FOLLOWER
I20260809 22:58:51.495075 27323 raft_consensus.cc:482] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:58:51.495067 27320 consensus_queue.cc:260] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.495235 27323 raft_consensus.cc:504] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.495949 27323 leader_election.cc:288] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194:45661), 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:51.496490 27320 ts_tablet_manager.cc:1428] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 22:58:51.496627 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2b6da779df44e389f533d4ebe6860f" candidate_uuid: "338b8db048e44dc29e1c35d4de03ae02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011" is_pre_election: true
I20260809 22:58:51.496894 27143 raft_consensus.cc:2454] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 338b8db048e44dc29e1c35d4de03ae02 in term 0.
I20260809 22:58:51.497217 26985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2b6da779df44e389f533d4ebe6860f" candidate_uuid: "338b8db048e44dc29e1c35d4de03ae02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ef5c5ae3b348da95df3408381bccc0" is_pre_election: true
I20260809 22:58:51.497435 26747 leader_election.cc:302] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2869045cf32547be8b5fd9082e247011, 338b8db048e44dc29e1c35d4de03ae02; no voters:
I20260809 22:58:51.497562 26985 raft_consensus.cc:2454] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 338b8db048e44dc29e1c35d4de03ae02 in term 0.
I20260809 22:58:51.497838 27323 raft_consensus.cc:2781] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:58:51.497974 27323 raft_consensus.cc:482] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:58:51.498090 27323 raft_consensus.cc:3037] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:51.500808 27323 raft_consensus.cc:504] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.501478 27323 leader_election.cc:288] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [CANDIDATE]: Term 1 election: Requested vote from peers e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194:45661), 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049)
I20260809 22:58:51.501940 26985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2b6da779df44e389f533d4ebe6860f" candidate_uuid: "338b8db048e44dc29e1c35d4de03ae02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1ef5c5ae3b348da95df3408381bccc0"
I20260809 22:58:51.502032 27143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee2b6da779df44e389f533d4ebe6860f" candidate_uuid: "338b8db048e44dc29e1c35d4de03ae02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2869045cf32547be8b5fd9082e247011"
I20260809 22:58:51.502218 26985 raft_consensus.cc:3037] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:51.502287 27143 raft_consensus.cc:3037] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:51.504601 27143 raft_consensus.cc:2454] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 338b8db048e44dc29e1c35d4de03ae02 in term 1.
I20260809 22:58:51.504673 26985 raft_consensus.cc:2454] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 338b8db048e44dc29e1c35d4de03ae02 in term 1.
I20260809 22:58:51.505088 26747 leader_election.cc:302] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2869045cf32547be8b5fd9082e247011, 338b8db048e44dc29e1c35d4de03ae02, e1ef5c5ae3b348da95df3408381bccc0; no voters:
I20260809 22:58:51.505494 27323 raft_consensus.cc:2781] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:51.505970 27323 raft_consensus.cc:686] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 1 LEADER]: Becoming Leader. State: Replica: 338b8db048e44dc29e1c35d4de03ae02, State: Running, Role: LEADER
I20260809 22:58:51.506392 27323 consensus_queue.cc:237] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } }
I20260809 22:58:51.509783 26641 catalog_manager.cc:5495] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193). New cstate: current_term: 1 leader_uuid: "338b8db048e44dc29e1c35d4de03ae02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "338b8db048e44dc29e1c35d4de03ae02" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 } health_report { overall_health: UNKNOWN } } }
I20260809 22:58:51.534991 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.529532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.24.31.193:35851)
0809 22:58:51.530032 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:51.530036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:51.530055 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:58:51.530132 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:51.530135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:51.530208 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:51.530352 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:51.530681 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.531467 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.532789 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.533267 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.533360 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.533768 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:51.534356 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:51.534461 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:51.534555 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 22:58:51.534621 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":287,"threads_started":1}
I20260809 22:58:51.549957 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.544479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:35361 (local address 127.24.31.194:45661)
0809 22:58:51.544702 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:51.544710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:51.545078 (+ 368us) server_negotiation.cc:407] Connection header received
0809 22:58:51.545344 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:51.545348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:51.545428 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:51.545531 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:51.546000 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.546735 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.548119 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.548552 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.548696 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.549116 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:51.549283 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:51.549383 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:51.549547 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 22:58:51.549628 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:58:51.550028 27344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.544338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:45661 (local address 127.24.31.193:35361)
0809 22:58:51.544971 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:51.544992 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:51.545195 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:51.545545 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:51.545550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:51.545562 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:58:51.545852 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:51.545860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.546872 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.546878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:51.547942 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:51.547955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.548731 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.548737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:51.548882 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.548931 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:58:51.549385 (+ 454us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:58:51.549406 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 22:58:51.549514 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 22:58:51.549645 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":302,"threads_started":1}
I20260809 22:58:51.558825 27143 raft_consensus.cc:1261] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Refusing update from remote peer 338b8db048e44dc29e1c35d4de03ae02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:51.559165 26985 raft_consensus.cc:1261] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Refusing update from remote peer 338b8db048e44dc29e1c35d4de03ae02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:58:51.559468 27339 consensus_queue.cc:1035] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2869045cf32547be8b5fd9082e247011" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:51.559938 27323 consensus_queue.cc:1035] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1ef5c5ae3b348da95df3408381bccc0" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:58:51.602905 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.587045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:46773 (local address 127.24.31.193:35851)
0809 22:58:51.587303 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:51.587310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:51.597685 (+ 10375us) server_negotiation.cc:407] Connection header received
0809 22:58:51.597844 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:51.597847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:51.597914 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:51.597987 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:58:51.598568 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.599370 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.600832 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.601437 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.601602 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.602148 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:58:51.602280 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:58:51.602379 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:51.602540 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 22:58:51.602619 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260809 22:58:51.603030 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:51.586974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:35851 (local address 127.24.31.194:46773)
0809 22:58:51.597579 (+ 10605us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:51.597596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:51.597712 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:51.598058 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:51.598062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:51.598071 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:58:51.598425 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:51.598436 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.599529 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.599532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:51.600688 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:51.600700 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:51.601741 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:51.601745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:51.601873 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:51.601907 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:58:51.602398 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:58:51.602408 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 22:58:51.602508 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 22:58:51.602644 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10467,"thread_start_us":10349,"threads_started":1}
W20260809 22:58:51.604333 27288 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 22:58:53.206820 26642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:53.207049 26642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:58:53.211166 26642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c94cad8671174defac7bd657b28c350c: Sending DeleteTablet for 3 replicas of tablet ee2b6da779df44e389f533d4ebe6860f
I20260809 22:58:53.211971 26799 tablet_service.cc:1514] Processing DeleteTablet for tablet ee2b6da779df44e389f533d4ebe6860f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:53 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:48568
I20260809 22:58:53.212108 26960 tablet_service.cc:1514] Processing DeleteTablet for tablet ee2b6da779df44e389f533d4ebe6860f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:53 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:34322
I20260809 22:58:53.212383 27112 tablet_service.cc:1514] Processing DeleteTablet for tablet ee2b6da779df44e389f533d4ebe6860f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:58:53 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:40344
I20260809 22:58:53.213104 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26726
I20260809 22:58:53.228375 27380 tablet_replica.cc:330] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: stopping tablet replica
I20260809 22:58:53.228689 27380 raft_consensus.cc:2229] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:53.228995 27380 raft_consensus.cc:2258] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:53.229538 27381 tablet_replica.cc:330] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: stopping tablet replica
I20260809 22:58:53.230309 27380 ts_tablet_manager.cc:1905] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:53.230365 27379 tablet_replica.cc:330] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: stopping tablet replica
I20260809 22:58:53.230369 27381 raft_consensus.cc:2229] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:58:53.230537 27379 raft_consensus.cc:2229] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:58:53.230832 27381 raft_consensus.cc:2258] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:53.231801 27379 raft_consensus.cc:2258] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:58:53.232398 27381 ts_tablet_manager.cc:1905] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:53.232715 27379 ts_tablet_manager.cc:1905] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:58:53.236991 27380 ts_tablet_manager.cc:1918] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:58:53.237128 27380 log.cc:1196] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/ee2b6da779df44e389f533d4ebe6860f
I20260809 22:58:53.237610 27380 ts_tablet_manager.cc:1939] T ee2b6da779df44e389f533d4ebe6860f P e1ef5c5ae3b348da95df3408381bccc0: Deleting consensus metadata
I20260809 22:58:53.238777 26623 catalog_manager.cc:4841] TS e1ef5c5ae3b348da95df3408381bccc0 (127.24.31.194:45661): tablet ee2b6da779df44e389f533d4ebe6860f (table test-table [id=3976a5136deb4cfa9426e5182944486d]) successfully deleted
I20260809 22:58:53.239439 27379 ts_tablet_manager.cc:1918] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:58:53.239439 27381 ts_tablet_manager.cc:1918] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 22:58:53.239598 27379 log.cc:1196] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/ee2b6da779df44e389f533d4ebe6860f
I20260809 22:58:53.239682 27381 log.cc:1196] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/ee2b6da779df44e389f533d4ebe6860f
I20260809 22:58:53.240221 27381 ts_tablet_manager.cc:1939] T ee2b6da779df44e389f533d4ebe6860f P 2869045cf32547be8b5fd9082e247011: Deleting consensus metadata
I20260809 22:58:53.240269 27379 ts_tablet_manager.cc:1939] T ee2b6da779df44e389f533d4ebe6860f P 338b8db048e44dc29e1c35d4de03ae02: Deleting consensus metadata
I20260809 22:58:53.241372 26626 catalog_manager.cc:4841] TS 2869045cf32547be8b5fd9082e247011 (127.24.31.195:36049): tablet ee2b6da779df44e389f533d4ebe6860f (table test-table [id=3976a5136deb4cfa9426e5182944486d]) successfully deleted
I20260809 22:58:53.241655 26623 catalog_manager.cc:4841] TS 338b8db048e44dc29e1c35d4de03ae02 (127.24.31.193:35851): tablet ee2b6da779df44e389f533d4ebe6860f (table test-table [id=3976a5136deb4cfa9426e5182944486d]) successfully deleted
Aug 09 22:58:53 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:58:53.254815 27383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:53.215469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48592 (local address 127.24.31.193:35851)
0809 22:58:53.229840 (+ 14371us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:53.229849 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:53.229880 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:58:53.229955 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:53.229963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:53.230082 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:53.231134 (+ 1052us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:53.232046 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:53.233233 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:53.239197 (+ 5964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:53.239979 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:53.241298 (+ 1319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:53.248831 (+ 7533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:53.248847 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:53.248863 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:53.248899 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:53.251864 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:53.252578 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:53.252587 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:53.252590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:53.252720 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:53.253144 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:53.253196 (+ 52us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:53.253200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:53.253564 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:53.253873 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:53.254192 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 22:58:53.254442 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14158,"thread_start_us":14018,"threads_started":1}
W20260809 22:58:53.255461 26873 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48592
W20260809 22:58:53.256054 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26726: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:53.256237 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 26726
I20260809 22:58:53.284046 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26883
Aug 09 22:58:53 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 22:58:53.299892 27384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:53.284934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34352 (local address 127.24.31.194:45661)
0809 22:58:53.285455 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:53.285459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:53.285476 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:58:53.285528 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:53.285532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:53.285589 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:53.285692 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:53.287041 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:53.287938 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:53.289493 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:53.289895 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:53.289999 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:53.294111 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:53.294125 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:53.294140 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:53.294182 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:53.297276 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:53.297977 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:53.297984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:53.297986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:53.298145 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:53.298542 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:53.298546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:53.298548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:53.298920 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:53.299146 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:53.299418 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 22:58:53.299585 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":305,"threads_started":1}
W20260809 22:58:53.300566 27032 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34352
W20260809 22:58:53.301257 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26883: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:53.301393 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 26883
I20260809 22:58:53.331369 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27042
Aug 09 22:58:53 dist-test-slave-vxj2 krb5kdc[26594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 22:58:53.346604 27385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:53.332179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40358 (local address 127.24.31.195:36049)
0809 22:58:53.332777 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:53.332781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:53.332797 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:58:53.332859 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:53.332863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:53.332920 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:53.333019 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:53.334457 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:53.335253 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:53.336448 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:53.336828 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:53.336926 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:53.341372 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:53.341386 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:53.341402 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:53.341436 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:53.344126 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:53.344730 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:53.344736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:53.344739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:53.344895 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:53.345285 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:53.345288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:53.345290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:53.345603 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:53.345822 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:53.346029 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 22:58:53.346227 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":306,"threads_started":1}
W20260809 22:58:53.347298 27185 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40358
W20260809 22:58:53.347954 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27042: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:58:53.348140 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 27042
I20260809 22:58:53.369598 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 26607
I20260809 22:58:53.370854 26689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:53.493814 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 26607
2026-08-09T22:58:53Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9054 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:53 dist-test-slave-vxj2 krb5kdc[27389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:53 dist-test-slave-vxj2 krb5kdc[27389](info): set up 2 sockets
Aug 09 22:58:53 dist-test-slave-vxj2 krb5kdc[27389](info): commencing operation
krb5kdc: starting...
W20260809 22:58:54.666920 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.004s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:58:54 dist-test-slave-vxj2 krb5kdc[27389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:58:54.873008 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42101
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:42101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:55.193293 27407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:55.193602 27407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:55.193802 27407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:55.201349 27407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:55.201475 27407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:55.201588 27407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:55.201689 27407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:55.201773 27407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:55.211947 27407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:42101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:55.213434 27407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:55.214869 27407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:55.227711 27413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:55.227684 27416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:55.227643 27414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:55.228452 27407 server_base.cc:1034] running on GCE node
I20260809 22:58:55.229285 27407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:55.230669 27407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:55.231882 27407 hybrid_clock.cc:648] HybridClock initialized: now 1786316335231838 us; error 36 us; skew 500 ppm
I20260809 22:58:55.232250 27407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:55 dist-test-slave-vxj2 krb5kdc[27389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316335, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:55.235458 27407 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:58:55.237061 27407 webserver.cc:460] Webserver started at http://127.24.31.254:41647/ using document root <none> and password file <none>
I20260809 22:58:55.237788 27407 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:55.237919 27407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:55.238262 27407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:55.241258 27407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "85226096725845a092fa77f286c34645"
format_stamp: "Formatted at 2026-08-09 22:58:55 on dist-test-slave-vxj2"
server_key: "2e536b832d4f10d80720fc02fe81d1ba"
server_key_iv: "b79ea9d07742a7a47fb490bca2f1e282"
server_key_version: "encryptionkey@0"
I20260809 22:58:55.242070 27407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "85226096725845a092fa77f286c34645"
format_stamp: "Formatted at 2026-08-09 22:58:55 on dist-test-slave-vxj2"
server_key: "2e536b832d4f10d80720fc02fe81d1ba"
server_key_iv: "b79ea9d07742a7a47fb490bca2f1e282"
server_key_version: "encryptionkey@0"
I20260809 22:58:55.247180 27407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:58:55.250686 27424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:55.251996 27407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:58:55.252174 27407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "85226096725845a092fa77f286c34645"
format_stamp: "Formatted at 2026-08-09 22:58:55 on dist-test-slave-vxj2"
server_key: "2e536b832d4f10d80720fc02fe81d1ba"
server_key_iv: "b79ea9d07742a7a47fb490bca2f1e282"
server_key_version: "encryptionkey@0"
I20260809 22:58:55.252393 27407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:55.278314 27407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:55.282514 27407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:55.282821 27407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:55.302129 27407 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:42101
I20260809 22:58:55.302160 27481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:42101 every 8 connection(s)
I20260809 22:58:55.303529 27407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:55.307349 27482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:55.311408 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 27407
I20260809 22:58:55.311655 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:58:55.312314 24703 external_mini_cluster.cc:1428] Setting key 047941a907653af22d0ad628d4abfb90
I20260809 22:58:55.318400 27482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645: Bootstrap starting.
I20260809 22:58:55.322100 27482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:55.323295 27482 log.cc:824] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:55.326872 27482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645: No bootstrap required, opened a new log
Aug 09 22:58:55 dist-test-slave-vxj2 krb5kdc[27389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:55.336418 27485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:55.314316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.24.31.254:42101)
0809 22:58:55.315130 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:55.315144 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:55.315190 (+ 46us) server_negotiation.cc:407] Connection header received
0809 22:58:55.316974 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:55.316995 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:55.317468 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:55.317775 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:55.319637 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:55.320962 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:55.322796 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:55.323543 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:55.323674 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:55.328856 (+ 5182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:55.328864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:55.328877 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:55.328925 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:55.332205 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:55.333002 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:55.333008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:55.333014 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:55.333247 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:55.333654 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:55.333657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:55.333659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:55.334169 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:55.334484 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:55.334972 (+ 488us) server_negotiation.cc:299] Negotiation successful
0809 22:58:55.335412 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":397,"threads_started":1}
I20260809 22:58:55.336597 27482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85226096725845a092fa77f286c34645" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42101 } }
I20260809 22:58:55.337013 27482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:55.337131 27482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85226096725845a092fa77f286c34645, State: Initialized, Role: FOLLOWER
I20260809 22:58:55.337780 27482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85226096725845a092fa77f286c34645" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42101 } }
I20260809 22:58:55.338044 27482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:55.338204 27482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:55.338394 27482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:55.340665 27482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85226096725845a092fa77f286c34645" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42101 } }
I20260809 22:58:55.341189 27482 leader_election.cc:302] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85226096725845a092fa77f286c34645; no voters:
I20260809 22:58:55.341796 27482 leader_election.cc:288] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:55.341974 27487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:55.343160 27487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [term 1 LEADER]: Becoming Leader. State: Replica: 85226096725845a092fa77f286c34645, State: Running, Role: LEADER
I20260809 22:58:55.343696 27487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85226096725845a092fa77f286c34645" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42101 } }
I20260809 22:58:55.344058 27482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:55.346726 27489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85226096725845a092fa77f286c34645. Latest consensus state: current_term: 1 leader_uuid: "85226096725845a092fa77f286c34645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85226096725845a092fa77f286c34645" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42101 } } }
I20260809 22:58:55.347190 27489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:55.347812 27488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85226096725845a092fa77f286c34645" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85226096725845a092fa77f286c34645" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42101 } } }
I20260809 22:58:55.348269 27488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:55.349084 27492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:55.353832 27492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:55.363217 27492 catalog_manager.cc:1347] Generated new cluster ID: 001f260a15794b65add1dde10db60616
I20260809 22:58:55.363335 27492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:55.378122 27492 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:55.380005 27492 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:55.387197 27492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85226096725845a092fa77f286c34645: Generated new TSK 0
I20260809 22:58:55.388180 27492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:58:55.440318 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42101
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:55.775152 27512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:55.775420 27512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:55.775514 27512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:55.782675 27512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:55.782793 27512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:55.782912 27512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:58:55.792718 27512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:55.793957 27512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:55.795252 27512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:55.808413 27518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:55.808418 27519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:55.808364 27521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:55.809305 27512 server_base.cc:1034] running on GCE node
I20260809 22:58:55.809880 27512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:55.810787 27512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:55.812029 27512 hybrid_clock.cc:648] HybridClock initialized: now 1786316335811983 us; error 46 us; skew 500 ppm
I20260809 22:58:55.812355 27512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:55 dist-test-slave-vxj2 krb5kdc[27389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316335, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:55.815507 27512 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:58:55.817034 27512 webserver.cc:460] Webserver started at http://127.24.31.193:34981/ using document root <none> and password file <none>
I20260809 22:58:55.817782 27512 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:55.817942 27512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:55.818262 27512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:55.821197 27512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "54c2768c8696465e94e11cd41533ffc7"
format_stamp: "Formatted at 2026-08-09 22:58:55 on dist-test-slave-vxj2"
server_key: "0a87c83171dd43d89c64dfb9f79c1567"
server_key_iv: "9ec13ba046fff6af0daaa5e41d655e29"
server_key_version: "encryptionkey@0"
I20260809 22:58:55.822026 27512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "54c2768c8696465e94e11cd41533ffc7"
format_stamp: "Formatted at 2026-08-09 22:58:55 on dist-test-slave-vxj2"
server_key: "0a87c83171dd43d89c64dfb9f79c1567"
server_key_iv: "9ec13ba046fff6af0daaa5e41d655e29"
server_key_version: "encryptionkey@0"
I20260809 22:58:55.827193 27512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:58:55.830652 27530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:55.831976 27512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:58:55.832182 27512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "54c2768c8696465e94e11cd41533ffc7"
format_stamp: "Formatted at 2026-08-09 22:58:55 on dist-test-slave-vxj2"
server_key: "0a87c83171dd43d89c64dfb9f79c1567"
server_key_iv: "9ec13ba046fff6af0daaa5e41d655e29"
server_key_version: "encryptionkey@0"
I20260809 22:58:55.832389 27512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:55.847561 27512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:55.851754 27512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:55.852048 27512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:55.852973 27512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:58:55.854310 27512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:55.854416 27512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:55.854562 27512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:55.854645 27512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:55.889487 27512 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:43957
I20260809 22:58:55.889523 27655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:43957 every 8 connection(s)
I20260809 22:58:55.890787 27512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:58:55.892197 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 27512
I20260809 22:58:55.892465 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:58:55.893343 24703 external_mini_cluster.cc:1428] Setting key 20ade21b5bf769f2b64ef593ddb63f4d
Aug 09 22:58:55 dist-test-slave-vxj2 krb5kdc[27389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316335, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:55.914368 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:55.893734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:57601 (local address 127.24.31.254:42101)
0809 22:58:55.894251 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:55.894257 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:55.895340 (+ 1083us) server_negotiation.cc:407] Connection header received
0809 22:58:55.897277 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:55.897286 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:55.897379 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:55.897504 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:55.899214 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:55.900377 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:55.902266 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:55.902989 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:55.903147 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:55.906837 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:55.906846 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:55.906855 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:55.906902 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:55.910446 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:55.911496 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:55.911505 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:55.911511 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:55.911680 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:55.912168 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:55.912176 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:55.912182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:55.912638 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:55.912813 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:55.913668 (+ 855us) server_negotiation.cc:299] Negotiation successful
0809 22:58:55.913938 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":431,"threads_started":1}
I20260809 22:58:55.915081 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:55.894314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:42101 (local address 127.24.31.193:57601)
0809 22:58:55.895147 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:55.895228 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:55.896979 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:55.897847 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:55.897860 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:55.898461 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:55.899028 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:55.899045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:55.900556 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:55.900560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:55.902033 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:55.902075 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:55.903220 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:55.903225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:55.903433 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:55.904133 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:55.904154 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:55.906599 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:55.910591 (+ 3992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:55.910598 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:55.910610 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:55.911286 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:55.911827 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:55.911831 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:55.911833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:55.912025 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:55.912800 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:55.912807 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:55.913245 (+ 438us) client_negotiation.cc:769] Sending connection context
0809 22:58:55.913504 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 22:58:55.913937 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":401,"threads_started":1}
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
I20260809 22:58:55.916754 27657 heartbeater.cc:347] Connected to a master server at 127.24.31.254:42101
I20260809 22:58:55.917060 27657 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:55.917826 27657 heartbeater.cc:511] Master 127.24.31.254:42101 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
I20260809 22:58:55.919950 27443 ts_manager.cc:194] Registered new tserver with Master: 54c2768c8696465e94e11cd41533ffc7 (127.24.31.193:43957)
I20260809 22:58:55.921470 27443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:57601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:58:55.943826 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42101
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:56.267020 27664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:56.267350 27664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:56.267524 27664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:56.274775 27664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:56.274914 27664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:56.275126 27664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:58:56.284682 27664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:56.286024 27664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:56.287350 27664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:56.300661 27669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:56.300737 27670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:56.300770 27673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:56.303429 27664 server_base.cc:1034] running on GCE node
I20260809 22:58:56.304024 27664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:56.305051 27664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:56.306268 27664 hybrid_clock.cc:648] HybridClock initialized: now 1786316336306220 us; error 62 us; skew 500 ppm
I20260809 22:58:56.306592 27664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:56 dist-test-slave-vxj2 krb5kdc[27389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316336, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:56.309793 27664 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:58:56.311429 27664 webserver.cc:460] Webserver started at http://127.24.31.194:36957/ using document root <none> and password file <none>
I20260809 22:58:56.312108 27664 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:56.312233 27664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:56.312549 27664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:56.315435 27664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "2d0f6503ad1c45099055468c35a972ab"
format_stamp: "Formatted at 2026-08-09 22:58:56 on dist-test-slave-vxj2"
server_key: "ef85768558e576292a6704b30e578907"
server_key_iv: "3d3d28ac9e43f45f4554fbacfa2c40ca"
server_key_version: "encryptionkey@0"
I20260809 22:58:56.316215 27664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "2d0f6503ad1c45099055468c35a972ab"
format_stamp: "Formatted at 2026-08-09 22:58:56 on dist-test-slave-vxj2"
server_key: "ef85768558e576292a6704b30e578907"
server_key_iv: "3d3d28ac9e43f45f4554fbacfa2c40ca"
server_key_version: "encryptionkey@0"
I20260809 22:58:56.321305 27664 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:58:56.324582 27681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:56.325903 27664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:58:56.326085 27664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "2d0f6503ad1c45099055468c35a972ab"
format_stamp: "Formatted at 2026-08-09 22:58:56 on dist-test-slave-vxj2"
server_key: "ef85768558e576292a6704b30e578907"
server_key_iv: "3d3d28ac9e43f45f4554fbacfa2c40ca"
server_key_version: "encryptionkey@0"
I20260809 22:58:56.326251 27664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:56.355525 27664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:56.359880 27664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:56.360183 27664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:56.361122 27664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:58:56.362476 27664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:56.362550 27664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:56.362633 27664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:56.362685 27664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:56.397442 27664 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:41861
I20260809 22:58:56.397467 27808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:41861 every 8 connection(s)
I20260809 22:58:56.399235 27664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:58:56.402380 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 27664
I20260809 22:58:56.402598 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:58:56.403204 24703 external_mini_cluster.cc:1428] Setting key c5af5caf72cf5c03004d2e99247da32d
Aug 09 22:58:56 dist-test-slave-vxj2 krb5kdc[27389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316336, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:56.421031 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:56.401806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:35089 (local address 127.24.31.254:42101)
0809 22:58:56.402077 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:56.402084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:56.403680 (+ 1596us) server_negotiation.cc:407] Connection header received
0809 22:58:56.405714 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:56.405720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:56.405802 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:56.405916 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:56.407374 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:56.408523 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:56.410113 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:56.410844 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:56.411001 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:56.415030 (+ 4029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:56.415035 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:56.415038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:56.415071 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:56.417813 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:56.418548 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:56.418552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:56.418554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:56.418697 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:56.419169 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:56.419173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:56.419175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:56.419462 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:56.419632 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:56.420418 (+ 786us) server_negotiation.cc:299] Negotiation successful
0809 22:58:56.420687 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 22:58:56.422008 27812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:56.402307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:42101 (local address 127.24.31.194:35089)
0809 22:58:56.403499 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:56.403546 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:56.405451 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:56.406115 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:56.406123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:56.406631 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:56.407187 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:56.407205 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:56.408670 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:56.408674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:56.409965 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:56.409977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:56.411075 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:56.411081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:56.411297 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:56.412056 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:56.412091 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:56.414794 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:56.417959 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:56.417968 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:56.417984 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:56.418404 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:56.418813 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:56.418818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:56.418822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:56.419041 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:56.419596 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:56.419604 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:56.420040 (+ 436us) client_negotiation.cc:769] Sending connection context
0809 22:58:56.420326 (+ 286us) client_negotiation.cc:240] Negotiation successful
0809 22:58:56.420866 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":399,"threads_started":1}
I20260809 22:58:56.423534 27809 heartbeater.cc:347] Connected to a master server at 127.24.31.254:42101
I20260809 22:58:56.423847 27809 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
I20260809 22:58:56.424600 27809 heartbeater.cc:511] Master 127.24.31.254:42101 requested a full tablet report, sending...
I20260809 22:58:56.426239 27443 ts_manager.cc:194] Registered new tserver with Master: 2d0f6503ad1c45099055468c35a972ab (127.24.31.194:41861)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
I20260809 22:58:56.427181 27443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:35089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:58:56.452811 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42101
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:56.775933 27816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:56.776257 27816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:56.776440 27816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:56.783638 27816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:56.783790 27816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:56.783998 27816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:58:56.793462 27816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:56.794723 27816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:56.796005 27816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:56.809722 27821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:56.809669 27825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:56.809758 27823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:56.810804 27816 server_base.cc:1034] running on GCE node
I20260809 22:58:56.811408 27816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:56.812309 27816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:56.813553 27816 hybrid_clock.cc:648] HybridClock initialized: now 1786316336813511 us; error 52 us; skew 500 ppm
I20260809 22:58:56.813876 27816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:56 dist-test-slave-vxj2 krb5kdc[27389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316336, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:56.816879 27816 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:58:56.818274 27816 webserver.cc:460] Webserver started at http://127.24.31.195:33233/ using document root <none> and password file <none>
I20260809 22:58:56.818881 27816 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:56.818985 27816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:56.819231 27816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:56.822106 27816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "cffde4db72754911916c8816ecbf2142"
format_stamp: "Formatted at 2026-08-09 22:58:56 on dist-test-slave-vxj2"
server_key: "1b12ff8f9d2b8d1d44862fd6488f06c3"
server_key_iv: "f63819577c46ec3caf42ec83346b91df"
server_key_version: "encryptionkey@0"
I20260809 22:58:56.822881 27816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "cffde4db72754911916c8816ecbf2142"
format_stamp: "Formatted at 2026-08-09 22:58:56 on dist-test-slave-vxj2"
server_key: "1b12ff8f9d2b8d1d44862fd6488f06c3"
server_key_iv: "f63819577c46ec3caf42ec83346b91df"
server_key_version: "encryptionkey@0"
I20260809 22:58:56.827878 27816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:58:56.831524 27833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:56.832813 27816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:58:56.832989 27816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "cffde4db72754911916c8816ecbf2142"
format_stamp: "Formatted at 2026-08-09 22:58:56 on dist-test-slave-vxj2"
server_key: "1b12ff8f9d2b8d1d44862fd6488f06c3"
server_key_iv: "f63819577c46ec3caf42ec83346b91df"
server_key_version: "encryptionkey@0"
I20260809 22:58:56.833220 27816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:56.852568 27816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:56.856566 27816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:56.856863 27816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:56.857848 27816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:58:56.859192 27816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:58:56.859269 27816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:56.859411 27816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:58:56.859473 27816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:56.894193 27816 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:44721
I20260809 22:58:56.894300 27954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:44721 every 8 connection(s)
I20260809 22:58:56.895439 27816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:58:56.900987 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 27816
I20260809 22:58:56.901237 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:58:56.901703 24703 external_mini_cluster.cc:1428] Setting key 3138d5a5b701a7376eac05fc62a52ce9
Aug 09 22:58:56 dist-test-slave-vxj2 krb5kdc[27389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316336, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:56.914718 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:56.898075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:50539 (local address 127.24.31.254:42101)
0809 22:58:56.898253 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:56.898257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:56.899315 (+ 1058us) server_negotiation.cc:407] Connection header received
0809 22:58:56.901385 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:56.901391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:56.901517 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:56.901636 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:56.903115 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:56.903886 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:56.905680 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:56.906127 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:56.906267 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:56.909665 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:56.909669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:56.909674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:56.909705 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:56.912104 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:56.912760 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:56.912764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:56.912766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:56.912900 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:56.913333 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:56.913336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:56.913338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:56.913593 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:56.913702 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:56.914324 (+ 622us) server_negotiation.cc:299] Negotiation successful
0809 22:58:56.914481 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:58:56.915745 27958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:56.898477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:42101 (local address 127.24.31.195:50539)
0809 22:58:56.899155 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 22:58:56.899192 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:58:56.901059 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:58:56.901904 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:58:56.901912 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:58:56.902407 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:58:56.902941 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:56.902960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:56.904048 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:56.904053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:56.905472 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:58:56.905485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:58:56.906360 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:58:56.906369 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:58:56.906619 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:56.907221 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:58:56.907247 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:58:56.909486 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:58:56.912227 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:56.912233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:56.912244 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:56.912653 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:56.912999 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:58:56.913002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:58:56.913004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:58:56.913233 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:58:56.913696 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:58:56.913702 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:58:56.913997 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 22:58:56.914219 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 22:58:56.914572 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":267,"threads_started":1}
I20260809 22:58:56.917284 27955 heartbeater.cc:347] Connected to a master server at 127.24.31.254:42101
I20260809 22:58:56.917587 27955 heartbeater.cc:464] Registering TS with master...
I20260809 22:58:56.918291 27955 heartbeater.cc:511] Master 127.24.31.254:42101 requested a full tablet report, sending...
I20260809 22:58:56.919711 27442 ts_manager.cc:194] Registered new tserver with Master: cffde4db72754911916c8816ecbf2142 (127.24.31.195:44721)
I20260809 22:58:56.920537 27442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:50539
I20260809 22:58:56.924829 27657 heartbeater.cc:503] Master 127.24.31.254:42101 was elected leader, sending a full tablet report...
I20260809 22:58:56.929289 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 22:58:56.964407 27660 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:58:56.961745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33034 (local address 127.24.31.254:42101)
0809 22:58:56.961924 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:56.961928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:56.962472 (+ 544us) server_negotiation.cc:407] Connection header received
0809 22:58:56.962704 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:56.962709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:56.962818 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:56.962935 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:56.964196 (+ 1261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33034: BlockingRecv error: recv got EOF from 127.0.0.1:33034 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:58:56.967906 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27512
W20260809 22:58:56.970791 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27512: Not authorized: Client connection negotiation failed: client connection to 127.24.31.193:43957: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:58:56.970990 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 27512
I20260809 22:58:56.989292 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27664
W20260809 22:58:56.992040 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27664: Not authorized: Client connection negotiation failed: client connection to 127.24.31.194:41861: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:58:56.992223 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 27664
I20260809 22:58:57.010499 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27816
W20260809 22:58:57.013476 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27816: Not authorized: Client connection negotiation failed: client connection to 127.24.31.195:44721: server requires authentication, but client does not have Kerberos credentials available
I20260809 22:58:57.013634 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 27816
I20260809 22:58:57.031919 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27407
I20260809 22:58:57.033241 27477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:57.139516 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 27407
2026-08-09T22:58:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3643 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:57 dist-test-slave-vxj2 krb5kdc[27976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:57 dist-test-slave-vxj2 krb5kdc[27976](info): set up 2 sockets
Aug 09 22:58:57 dist-test-slave-vxj2 krb5kdc[27976](info): commencing operation
krb5kdc: starting...
W20260809 22:58:58.307217 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:58:58 dist-test-slave-vxj2 krb5kdc[27976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:58:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:58:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:58:58.512943 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:38573
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:38573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:58:58.834909 27991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:58:58.835228 27991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:58:58.835394 27991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:58:58.842406 27991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:58:58.842566 27991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:58:58.842674 27991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:58:58.842777 27991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:58:58.842857 27991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:58:58.852351 27991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:38573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:38573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:58:58.853682 27991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:58:58.854984 27991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:58:58.866518 28000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:58.866632 27997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:58:58.866657 27998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:58:58.867578 27991 server_base.cc:1034] running on GCE node
I20260809 22:58:58.868304 27991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:58:58.869643 27991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:58:58.870844 27991 hybrid_clock.cc:648] HybridClock initialized: now 1786316338870808 us; error 27 us; skew 500 ppm
I20260809 22:58:58.871187 27991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:58:58 dist-test-slave-vxj2 krb5kdc[27976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316338, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:58:58.878999 27991 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:58:58.880496 27991 webserver.cc:460] Webserver started at http://127.24.31.254:46499/ using document root <none> and password file <none>
I20260809 22:58:58.881230 27991 fs_manager.cc:359] Metadata directory not provided
I20260809 22:58:58.881377 27991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:58:58.881696 27991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:58:58.884547 27991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "90e188a2fc8845eb89baababe9a6b6e7"
format_stamp: "Formatted at 2026-08-09 22:58:58 on dist-test-slave-vxj2"
server_key: "895dd09bcf48cbf0b1535fee9c13f483"
server_key_iv: "131aff7a4dd3f56476ffa63702042be4"
server_key_version: "encryptionkey@0"
I20260809 22:58:58.885396 27991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "90e188a2fc8845eb89baababe9a6b6e7"
format_stamp: "Formatted at 2026-08-09 22:58:58 on dist-test-slave-vxj2"
server_key: "895dd09bcf48cbf0b1535fee9c13f483"
server_key_iv: "131aff7a4dd3f56476ffa63702042be4"
server_key_version: "encryptionkey@0"
I20260809 22:58:58.890322 27991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:58:58.893592 28007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:58:58.894805 27991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 22:58:58.894994 27991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "90e188a2fc8845eb89baababe9a6b6e7"
format_stamp: "Formatted at 2026-08-09 22:58:58 on dist-test-slave-vxj2"
server_key: "895dd09bcf48cbf0b1535fee9c13f483"
server_key_iv: "131aff7a4dd3f56476ffa63702042be4"
server_key_version: "encryptionkey@0"
I20260809 22:58:58.895185 27991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:58:58.913947 27991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:58:58.918314 27991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:58:58.918624 27991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:58:58.937305 27991 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:38573
I20260809 22:58:58.937389 28063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:38573 every 8 connection(s)
I20260809 22:58:58.938710 27991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:58:58.942258 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 27991
I20260809 22:58:58.942553 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:58:58.943382 28064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:58:58.943578 24703 external_mini_cluster.cc:1428] Setting key a377fab1e562e1da9b7975c4b639dea9
I20260809 22:58:58.958863 28064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7: Bootstrap starting.
Aug 09 22:58:58 dist-test-slave-vxj2 krb5kdc[27976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:58:58.962175 28064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7: Neither blocks nor log segments found. Creating new log.
I20260809 22:58:58.963223 28064 log.cc:824] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7: Log is configured to *not* fsync() on all Append() calls
I20260809 22:58:58.965966 28064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7: No bootstrap required, opened a new log
I20260809 22:58:58.967396 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:58.945626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.24.31.254:38573)
0809 22:58:58.946438 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:58.946451 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:58.946504 (+ 53us) server_negotiation.cc:407] Connection header received
0809 22:58:58.948842 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:58.948865 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:58.949304 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:58.949626 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:58.951513 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:58.952792 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:58.954562 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:58.955248 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:58.955379 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:58.960141 (+ 4762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:58.960149 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:58.960165 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:58:58.960205 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:58.963430 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:58.964119 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:58.964126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:58.964131 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:58.964314 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:58:58.964718 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:58:58.964722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:58:58.964724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:58:58.965285 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:58.965591 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:58.966051 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 22:58:58.966457 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":392,"threads_started":1}
I20260809 22:58:58.971228 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:58:58.973999 28064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e188a2fc8845eb89baababe9a6b6e7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38573 } }
I20260809 22:58:58.974354 28064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:58:58.974457 28064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e188a2fc8845eb89baababe9a6b6e7, State: Initialized, Role: FOLLOWER
I20260809 22:58:58.974989 28064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e188a2fc8845eb89baababe9a6b6e7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38573 } }
I20260809 22:58:58.975229 28064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:58:58.975383 28064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:58:58.975553 28064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:58:58.977736 28064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e188a2fc8845eb89baababe9a6b6e7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38573 } }
I20260809 22:58:58.978240 28064 leader_election.cc:302] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90e188a2fc8845eb89baababe9a6b6e7; no voters:
I20260809 22:58:58.978952 28064 leader_election.cc:288] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:58:58.979020 28071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:58:58.981125 28064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:58:58.981451 28071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [term 1 LEADER]: Becoming Leader. State: Replica: 90e188a2fc8845eb89baababe9a6b6e7, State: Running, Role: LEADER
I20260809 22:58:58.982007 28071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e188a2fc8845eb89baababe9a6b6e7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38573 } }
I20260809 22:58:58.984941 28072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90e188a2fc8845eb89baababe9a6b6e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e188a2fc8845eb89baababe9a6b6e7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38573 } } }
I20260809 22:58:58.985436 28072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:58.985889 28073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90e188a2fc8845eb89baababe9a6b6e7. Latest consensus state: current_term: 1 leader_uuid: "90e188a2fc8845eb89baababe9a6b6e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90e188a2fc8845eb89baababe9a6b6e7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38573 } } }
I20260809 22:58:58.986155 28073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7 [sys.catalog]: This master's current role is: LEADER
I20260809 22:58:58.987859 28081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:58:58.993208 28081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:58:59.018636 28081 catalog_manager.cc:1347] Generated new cluster ID: 154188de856c4544a57c2c7a841447bc
I20260809 22:58:59.018770 28081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:58:59.030021 28093 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:58:59.033313 28081 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:58:59.035051 28081 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:58:59.041548 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:58:59.033182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36840 (local address 127.24.31.254:38573)
0809 22:58:59.033411 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 22:58:59.033420 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:58:59.034302 (+ 882us) server_negotiation.cc:407] Connection header received
0809 22:58:59.034508 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:58:59.034517 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:58:59.035051 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:58:59.035178 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:58:59.036275 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:59.036957 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:59.038295 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:58:59.038723 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:58:59.038858 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:58:59.039963 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:58:59.039968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:58:59.039971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:58:59.040596 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:58:59.040637 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:58:59.040663 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:58:59.040793 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 22:58:59.040969 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 22:58:59.041058 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:58:59.051092 28081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90e188a2fc8845eb89baababe9a6b6e7: Generated new TSK 0
I20260809 22:58:59.051906 28081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:58:59.060921 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 27991
I20260809 22:58:59.062114 28059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:58:59.151955 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 27991
2026-08-09T22:58:59Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2006 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:58:59 dist-test-slave-vxj2 krb5kdc[28102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:58:59 dist-test-slave-vxj2 krb5kdc[28102](info): set up 2 sockets
Aug 09 22:58:59 dist-test-slave-vxj2 krb5kdc[28102](info): commencing operation
krb5kdc: starting...
W20260809 22:59:00.306888 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.005s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:59:00 dist-test-slave-vxj2 krb5kdc[28102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:59:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:59:00.520790 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:37499
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:36887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:37499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:00.846443 28118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:00.846733 28118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:00.846828 28118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:00.853946 28118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:00.854048 28118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:00.854099 28118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:00.854156 28118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:00.854192 28118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:00.863839 28118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:37499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:37499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:00.865054 28118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:00.866350 28118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:00.877892 28123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:00.877916 28124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:00.877916 28127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:00.878899 28118 server_base.cc:1034] running on GCE node
I20260809 22:59:00.879580 28118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:00.880854 28118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:00.882055 28118 hybrid_clock.cc:648] HybridClock initialized: now 1786316340882004 us; error 35 us; skew 500 ppm
I20260809 22:59:00.882405 28118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:00 dist-test-slave-vxj2 krb5kdc[28102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316340, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:00.885474 28118 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:59:00.886981 28118 webserver.cc:460] Webserver started at http://127.24.31.254:34047/ using document root <none> and password file <none>
I20260809 22:59:00.887648 28118 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:00.887794 28118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:00.888098 28118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:00.891055 28118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "3b28de9514484898ba47f6ffc6b47b24"
format_stamp: "Formatted at 2026-08-09 22:59:00 on dist-test-slave-vxj2"
server_key: "14892c15721176e4179de1579efa8406"
server_key_iv: "92ea65680f1b639435a0d91af8e4e4c5"
server_key_version: "encryptionkey@0"
I20260809 22:59:00.891826 28118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "3b28de9514484898ba47f6ffc6b47b24"
format_stamp: "Formatted at 2026-08-09 22:59:00 on dist-test-slave-vxj2"
server_key: "14892c15721176e4179de1579efa8406"
server_key_iv: "92ea65680f1b639435a0d91af8e4e4c5"
server_key_version: "encryptionkey@0"
I20260809 22:59:00.896795 28118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 22:59:00.900082 28134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:00.901473 28118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:59:00.901654 28118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "3b28de9514484898ba47f6ffc6b47b24"
format_stamp: "Formatted at 2026-08-09 22:59:00 on dist-test-slave-vxj2"
server_key: "14892c15721176e4179de1579efa8406"
server_key_iv: "92ea65680f1b639435a0d91af8e4e4c5"
server_key_version: "encryptionkey@0"
I20260809 22:59:00.901856 28118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:00.919679 28118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:00.923583 28118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:00.923875 28118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:00.942314 28118 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:37499
I20260809 22:59:00.942348 28192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:37499 every 8 connection(s)
I20260809 22:59:00.943681 28118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:00.947319 28193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:00.949769 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 28118
I20260809 22:59:00.950002 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:00.950850 24703 external_mini_cluster.cc:1428] Setting key 3ea3063f583b5cce3db7cb7db4d0ae2c
I20260809 22:59:00.959318 28193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24: Bootstrap starting.
I20260809 22:59:00.963125 28193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:00.964450 28193 log.cc:824] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24: Log is configured to *not* fsync() on all Append() calls
Aug 09 22:59:00 dist-test-slave-vxj2 krb5kdc[28102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:00.967600 28193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24: No bootstrap required, opened a new log
I20260809 22:59:00.974406 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:00.952740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.24.31.254:37499)
0809 22:59:00.953522 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:00.953533 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:00.953571 (+ 38us) server_negotiation.cc:407] Connection header received
0809 22:59:00.955259 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:00.955280 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:00.955635 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:00.955935 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:00.957750 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:00.959037 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:00.960820 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:00.961584 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:00.961726 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:00.966900 (+ 5174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:00.966908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:00.966922 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:00.966955 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:00.970301 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:00.971006 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:00.971015 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:00.971022 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:00.971199 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:00.971584 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:00.971591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:00.971599 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:00.972125 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:00.972465 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:00.972893 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 22:59:00.973405 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":403,"threads_started":1}
I20260809 22:59:00.975814 28193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b28de9514484898ba47f6ffc6b47b24" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37499 } }
I20260809 22:59:00.976265 28193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:00.976379 28193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b28de9514484898ba47f6ffc6b47b24, State: Initialized, Role: FOLLOWER
I20260809 22:59:00.977003 28193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b28de9514484898ba47f6ffc6b47b24" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37499 } }
I20260809 22:59:00.977355 28193 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:00.977507 28193 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:00.977696 28193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:00.979905 28193 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b28de9514484898ba47f6ffc6b47b24" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37499 } }
I20260809 22:59:00.980409 28193 leader_election.cc:302] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b28de9514484898ba47f6ffc6b47b24; no voters:
I20260809 22:59:00.981017 28193 leader_election.cc:288] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:00.981187 28198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:00.982671 28198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [term 1 LEADER]: Becoming Leader. State: Replica: 3b28de9514484898ba47f6ffc6b47b24, State: Running, Role: LEADER
I20260809 22:59:00.983264 28193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:00.983207 28198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b28de9514484898ba47f6ffc6b47b24" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37499 } }
I20260809 22:59:00.986120 28201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b28de9514484898ba47f6ffc6b47b24. Latest consensus state: current_term: 1 leader_uuid: "3b28de9514484898ba47f6ffc6b47b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b28de9514484898ba47f6ffc6b47b24" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37499 } } }
I20260809 22:59:00.986502 28201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:00.986227 28200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b28de9514484898ba47f6ffc6b47b24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b28de9514484898ba47f6ffc6b47b24" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37499 } } }
I20260809 22:59:00.986693 28200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:00.988327 28205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:00.993513 28205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:01.003057 28205 catalog_manager.cc:1347] Generated new cluster ID: a78e23072de44007b1f46f1ce80d57b7
I20260809 22:59:01.003180 28205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:01.021951 28205 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:01.023269 28205 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:01.031567 28205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b28de9514484898ba47f6ffc6b47b24: Generated new TSK 0
I20260809 22:59:01.032451 28205 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:59:01.092420 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37499
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:36887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:01.440696 28222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:01.440971 28222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:01.441128 28222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:01.448459 28222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:01.448621 28222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:01.448872 28222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:01.458760 28222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:01.460103 28222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:01.461473 28222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:01.475337 28228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:01.475471 28227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:01.475469 28230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:01.476418 28222 server_base.cc:1034] running on GCE node
I20260809 22:59:01.477083 28222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:01.478098 28222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:01.479321 28222 hybrid_clock.cc:648] HybridClock initialized: now 1786316341479263 us; error 54 us; skew 500 ppm
I20260809 22:59:01.479665 28222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:01 dist-test-slave-vxj2 krb5kdc[28102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316341, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:01.483002 28222 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:59:01.484645 28222 webserver.cc:460] Webserver started at http://127.24.31.193:41001/ using document root <none> and password file <none>
I20260809 22:59:01.485487 28222 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:01.485636 28222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:01.485960 28222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:01.489319 28222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "a14bc4382bfb40d6a7557f9859e39c09"
format_stamp: "Formatted at 2026-08-09 22:59:01 on dist-test-slave-vxj2"
server_key: "45abeec646b998fce01320d00073200f"
server_key_iv: "27d3f74129e2376bdda8154cb0cda2c9"
server_key_version: "encryptionkey@0"
I20260809 22:59:01.490183 28222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "a14bc4382bfb40d6a7557f9859e39c09"
format_stamp: "Formatted at 2026-08-09 22:59:01 on dist-test-slave-vxj2"
server_key: "45abeec646b998fce01320d00073200f"
server_key_iv: "27d3f74129e2376bdda8154cb0cda2c9"
server_key_version: "encryptionkey@0"
I20260809 22:59:01.495383 28222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:59:01.498925 28239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:01.500316 28222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:59:01.500507 28222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "a14bc4382bfb40d6a7557f9859e39c09"
format_stamp: "Formatted at 2026-08-09 22:59:01 on dist-test-slave-vxj2"
server_key: "45abeec646b998fce01320d00073200f"
server_key_iv: "27d3f74129e2376bdda8154cb0cda2c9"
server_key_version: "encryptionkey@0"
I20260809 22:59:01.500715 28222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:01.537002 28222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:01.541100 28222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:01.541451 28222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:01.542420 28222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:01.543763 28222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:01.543870 28222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:01.544039 28222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:01.544117 28222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:01.578258 28222 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:34645
I20260809 22:59:01.578305 28369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:34645 every 8 connection(s)
I20260809 22:59:01.579581 28222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:01.587347 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 28222
I20260809 22:59:01.587567 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:59:01.588279 24703 external_mini_cluster.cc:1428] Setting key 6f81c4ec6c93b2d6ca390afa2a590a25
Aug 09 22:59:01 dist-test-slave-vxj2 krb5kdc[28102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316341, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:01.599716 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:01.582148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:32901 (local address 127.24.31.254:37499)
0809 22:59:01.582610 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:01.582614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:01.583479 (+ 865us) server_negotiation.cc:407] Connection header received
0809 22:59:01.585319 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:01.585324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:01.585424 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:01.585549 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:01.587037 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:01.587881 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:01.589696 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:01.590118 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:01.590248 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:01.593959 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:01.593964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:01.593967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:01.593998 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:01.596641 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:01.597448 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:01.597455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:01.597464 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:01.597576 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:01.598006 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:01.598011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:01.598016 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:01.598273 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:01.598431 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:01.599119 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 22:59:01.599340 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":255,"threads_started":1}
I20260809 22:59:01.600642 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:01.582619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:37499 (local address 127.24.31.193:32901)
0809 22:59:01.583313 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:01.583349 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:01.584995 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:01.585769 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:01.585777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:01.586306 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:01.586857 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:01.586873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:01.588052 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:01.588055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:01.589478 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:01.589498 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:01.590356 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:01.590363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:01.590596 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:01.591269 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:01.591291 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:01.593736 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:01.596784 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:01.596793 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:01.596808 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:01.597319 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:01.597690 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:01.597695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:01.597698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:01.597886 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:01.598412 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:01.598419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:01.598752 (+ 333us) client_negotiation.cc:769] Sending connection context
0809 22:59:01.599014 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 22:59:01.599546 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":315,"threads_started":1}
I20260809 22:59:01.602146 28370 heartbeater.cc:347] Connected to a master server at 127.24.31.254:37499
I20260809 22:59:01.602468 28370 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:01.603354 28370 heartbeater.cc:511] Master 127.24.31.254:37499 requested a full tablet report, sending...
I20260809 22:59:01.605772 28154 ts_manager.cc:194] Registered new tserver with Master: a14bc4382bfb40d6a7557f9859e39c09 (127.24.31.193:34645)
I20260809 22:59:01.607250 28154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:32901
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:59:01.636161 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37499
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:36887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:01.977421 28376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:01.977759 28376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:01.977949 28376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:01.985379 28376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:01.985566 28376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:01.985750 28376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:01.995998 28376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:01.997397 28376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:01.998893 28376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:02.012434 28384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:02.012530 28381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:02.012506 28382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:02.013473 28376 server_base.cc:1034] running on GCE node
I20260809 22:59:02.014179 28376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:02.015245 28376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:02.016479 28376 hybrid_clock.cc:648] HybridClock initialized: now 1786316342016433 us; error 52 us; skew 500 ppm
I20260809 22:59:02.016821 28376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:02 dist-test-slave-vxj2 krb5kdc[28102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:02.020112 28376 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:59:02.021679 28376 webserver.cc:460] Webserver started at http://127.24.31.194:41871/ using document root <none> and password file <none>
I20260809 22:59:02.022322 28376 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:02.022428 28376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:02.022701 28376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:02.025701 28376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "3038f7ac47dd412797f423a3a180f7d2"
format_stamp: "Formatted at 2026-08-09 22:59:02 on dist-test-slave-vxj2"
server_key: "ae2d732a8827e9e0b1cae2dc267b8d46"
server_key_iv: "262e7d1231ce94386436d4dbe1497279"
server_key_version: "encryptionkey@0"
I20260809 22:59:02.026495 28376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "3038f7ac47dd412797f423a3a180f7d2"
format_stamp: "Formatted at 2026-08-09 22:59:02 on dist-test-slave-vxj2"
server_key: "ae2d732a8827e9e0b1cae2dc267b8d46"
server_key_iv: "262e7d1231ce94386436d4dbe1497279"
server_key_version: "encryptionkey@0"
I20260809 22:59:02.031653 28376 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 22:59:02.035172 28392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:02.036574 28376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:59:02.036741 28376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "3038f7ac47dd412797f423a3a180f7d2"
format_stamp: "Formatted at 2026-08-09 22:59:02 on dist-test-slave-vxj2"
server_key: "ae2d732a8827e9e0b1cae2dc267b8d46"
server_key_iv: "262e7d1231ce94386436d4dbe1497279"
server_key_version: "encryptionkey@0"
I20260809 22:59:02.036939 28376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:02.066486 28376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:02.070861 28376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:02.071174 28376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:02.072175 28376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:02.073632 28376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:02.073707 28376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:02.073830 28376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:02.073889 28376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:02.109086 28376 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:37875
I20260809 22:59:02.109128 28516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:37875 every 8 connection(s)
I20260809 22:59:02.110441 28376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:02.116883 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 28376
I20260809 22:59:02.117087 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:59:02.117623 24703 external_mini_cluster.cc:1428] Setting key 84075900a20dc3ca9be0c8f60c51a76c
Aug 09 22:59:02 dist-test-slave-vxj2 krb5kdc[28102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:02.130725 28520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:02.113006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:54357 (local address 127.24.31.254:37499)
0809 22:59:02.113562 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:02.113567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:02.114463 (+ 896us) server_negotiation.cc:407] Connection header received
0809 22:59:02.116259 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:02.116265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:02.116356 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:02.116487 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:02.118157 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.119057 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.120606 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.121034 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.121210 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:02.124656 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:02.124662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:02.124665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:02.124702 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:02.127568 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.128307 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.128312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.128322 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.128456 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.128916 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.128920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.128922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.129252 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:02.129407 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:02.130089 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 22:59:02.130318 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":323,"threads_started":1}
I20260809 22:59:02.131685 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:02.113542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:37499 (local address 127.24.31.194:54357)
0809 22:59:02.114295 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:02.114334 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:02.115992 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:02.116712 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:02.116720 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:02.117375 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:02.117983 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:02.118003 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.119227 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.119231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:02.120455 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:02.120469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.121289 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.121298 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:02.121533 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:02.122234 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:02.122258 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:02.124418 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:02.127707 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:02.127715 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:02.127731 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:02.128171 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:02.128573 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:02.128578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:02.128582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:02.128779 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:02.129378 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:02.129386 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:02.129725 (+ 339us) client_negotiation.cc:769] Sending connection context
0809 22:59:02.129993 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 22:59:02.130524 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":277,"threads_started":1}
I20260809 22:59:02.133208 28517 heartbeater.cc:347] Connected to a master server at 127.24.31.254:37499
I20260809 22:59:02.133565 28517 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:02.134557 28517 heartbeater.cc:511] Master 127.24.31.254:37499 requested a full tablet report, sending...
I20260809 22:59:02.136281 28154 ts_manager.cc:194] Registered new tserver with Master: 3038f7ac47dd412797f423a3a180f7d2 (127.24.31.194:37875)
I20260809 22:59:02.137523 28154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:54357
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:59:02.165027 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37499
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:36887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:02.499235 28524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:02.499554 28524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:02.499661 28524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:02.508133 28524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:02.508267 28524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:02.508407 28524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:02.519234 28524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:02.520550 28524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:02.521903 28524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:02.536607 28530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:02.536545 28531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:02.536576 28535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:02.537510 28524 server_base.cc:1034] running on GCE node
I20260809 22:59:02.538198 28524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:02.545331 28524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:02.546599 28524 hybrid_clock.cc:648] HybridClock initialized: now 1786316342546521 us; error 69 us; skew 500 ppm
I20260809 22:59:02.547015 28524 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:02 dist-test-slave-vxj2 krb5kdc[28102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:02.551040 28524 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:59:02.552835 28524 webserver.cc:460] Webserver started at http://127.24.31.195:37635/ using document root <none> and password file <none>
I20260809 22:59:02.553766 28524 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:02.553876 28524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:02.554175 28524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:02.557431 28524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "32cb6ee1a0404e5dafb5145b3da6ab83"
format_stamp: "Formatted at 2026-08-09 22:59:02 on dist-test-slave-vxj2"
server_key: "4183d71f7b91640ee7eafc009e7d85de"
server_key_iv: "e95db1958eccff52cad2bc5d4163ae98"
server_key_version: "encryptionkey@0"
I20260809 22:59:02.558425 28524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "32cb6ee1a0404e5dafb5145b3da6ab83"
format_stamp: "Formatted at 2026-08-09 22:59:02 on dist-test-slave-vxj2"
server_key: "4183d71f7b91640ee7eafc009e7d85de"
server_key_iv: "e95db1958eccff52cad2bc5d4163ae98"
server_key_version: "encryptionkey@0"
I20260809 22:59:02.564015 28524 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 22:59:02.567549 28545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:02.568886 28524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:59:02.569085 28524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "32cb6ee1a0404e5dafb5145b3da6ab83"
format_stamp: "Formatted at 2026-08-09 22:59:02 on dist-test-slave-vxj2"
server_key: "4183d71f7b91640ee7eafc009e7d85de"
server_key_iv: "e95db1958eccff52cad2bc5d4163ae98"
server_key_version: "encryptionkey@0"
I20260809 22:59:02.569355 28524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:02.592269 28524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:02.596501 28524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:02.596805 28524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:02.597879 28524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:02.599216 28524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:02.599362 28524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:02.599509 28524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:02.599571 28524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:02.610226 28370 heartbeater.cc:503] Master 127.24.31.254:37499 was elected leader, sending a full tablet report...
I20260809 22:59:02.635929 28524 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:44973
I20260809 22:59:02.635984 28667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:44973 every 8 connection(s)
I20260809 22:59:02.637387 28524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:02.646929 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 28524
I20260809 22:59:02.647092 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:59:02.647572 24703 external_mini_cluster.cc:1428] Setting key 6ba9fd3551bb4e24cdc0d62ab457aff4
Aug 09 22:59:02 dist-test-slave-vxj2 krb5kdc[28102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:02.657783 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:02.639945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:44575 (local address 127.24.31.254:37499)
0809 22:59:02.640456 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:02.640462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:02.641392 (+ 930us) server_negotiation.cc:407] Connection header received
0809 22:59:02.643237 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:02.643243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:02.643356 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:02.643478 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:02.644974 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.645820 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.647772 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.648310 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.648482 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:02.652199 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:02.652210 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:02.652213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:02.652246 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:02.654754 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.655455 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.655463 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.655469 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.655593 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.656018 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.656024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.656027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.656292 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:02.656469 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:02.657132 (+ 663us) server_negotiation.cc:299] Negotiation successful
0809 22:59:02.657396 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":298,"threads_started":1}
I20260809 22:59:02.658689 28671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:02.640447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:37499 (local address 127.24.31.195:44575)
0809 22:59:02.641233 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:02.641273 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:02.642965 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:02.643718 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:02.643727 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:02.644238 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:02.644802 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:02.644819 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.645984 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.645988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:02.647480 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:02.647492 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.648564 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.648573 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:02.648822 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:02.649657 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:02.649696 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:02.652000 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:02.654903 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:02.654911 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:02.654934 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:02.655333 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:02.655707 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:02.655712 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:02.655716 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:02.655905 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:02.656428 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:02.656436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:02.656772 (+ 336us) client_negotiation.cc:769] Sending connection context
0809 22:59:02.657048 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 22:59:02.657589 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":273,"threads_started":1}
I20260809 22:59:02.659994 28668 heartbeater.cc:347] Connected to a master server at 127.24.31.254:37499
I20260809 22:59:02.660300 28668 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:02.661073 28668 heartbeater.cc:511] Master 127.24.31.254:37499 requested a full tablet report, sending...
I20260809 22:59:02.662753 28154 ts_manager.cc:194] Registered new tserver with Master: 32cb6ee1a0404e5dafb5145b3da6ab83 (127.24.31.195:44973)
I20260809 22:59:02.663630 28154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:44575
I20260809 22:59:02.675580 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:59:02 dist-test-slave-vxj2 krb5kdc[28102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316342, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 22:59:02 dist-test-slave-vxj2 krb5kdc[28102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316342, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:02.727674 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:02.713313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.24.31.254:37499)
0809 22:59:02.713491 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:02.713516 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:02.714017 (+ 501us) server_negotiation.cc:407] Connection header received
0809 22:59:02.714246 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:02.714251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:02.714325 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:02.714439 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:02.716065 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.716821 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.718146 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.718558 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.718695 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:02.722541 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:02.722546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:02.722549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:02.722583 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:02.725004 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.725672 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.725678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.725688 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.725811 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.726200 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.726203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.726205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.726531 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:02.726809 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:02.727123 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 22:59:02.727349 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 22:59:02.728231 28154 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58834
I20260809 22:59:02.730696 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28222
Aug 09 22:59:02 dist-test-slave-vxj2 krb5kdc[28102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316342, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:59:02.747181 28682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:02.731799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.24.31.193:34645)
0809 22:59:02.732418 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:02.732425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:02.732455 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:59:02.732512 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:02.732519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:02.732730 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:02.732897 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:02.734174 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.735205 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.736414 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.737002 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.737154 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:02.741278 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:02.741291 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:02.741306 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:02.741344 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:02.744214 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.744971 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.744979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.744981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.745109 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.745531 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.745537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.745540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.745905 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:02.746239 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:02.746526 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 22:59:02.746814 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":305,"threads_started":1}
W20260809 22:59:02.748601 28365 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55188
W20260809 22:59:02.749801 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28222: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:59:02.750037 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 28222
I20260809 22:59:02.768975 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28376
Aug 09 22:59:02 dist-test-slave-vxj2 krb5kdc[28102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316342, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 22:59:02.785872 28683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:02.769991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60558 (local address 127.24.31.194:37875)
0809 22:59:02.770624 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:02.770631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:02.770657 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:59:02.770722 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:02.770729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:02.770964 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:02.771097 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:02.772353 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.773533 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.774915 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.775477 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.775578 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:02.780274 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:02.780287 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:02.780301 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:02.780340 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:02.783104 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.783844 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.783851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.783853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.783973 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.784317 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.784320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.784322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.784653 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:02.784882 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:02.785226 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 22:59:02.785437 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":334,"threads_started":1}
W20260809 22:59:02.787163 28512 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60558
W20260809 22:59:02.788292 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28376: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:59:02.788450 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 28376
I20260809 22:59:02.806849 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28524
Aug 09 22:59:02 dist-test-slave-vxj2 krb5kdc[28102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316342, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 22:59:02.823846 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:02.807788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40560 (local address 127.24.31.195:44973)
0809 22:59:02.808369 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:02.808376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:02.808413 (+ 37us) server_negotiation.cc:407] Connection header received
0809 22:59:02.808478 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:02.808484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:02.808699 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:02.808877 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:02.810345 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.811395 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.812663 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:02.813333 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:02.813484 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:02.818085 (+ 4601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:02.818097 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:02.818112 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:02.818146 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:02.820795 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.821642 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.821650 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.821652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.821776 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:02.822175 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:02.822179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:02.822181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:02.822547 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:02.822874 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:02.823199 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 22:59:02.823495 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":277,"threads_started":1}
W20260809 22:59:02.825142 28662 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40560
W20260809 22:59:02.826311 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28524: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 22:59:02.826501 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 28524
I20260809 22:59:02.844269 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28118
I20260809 22:59:02.845453 28186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:02.951632 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 28118
2026-08-09T22:59:02Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3805 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:59:03 dist-test-slave-vxj2 krb5kdc[28689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:59:03 dist-test-slave-vxj2 krb5kdc[28689](info): set up 2 sockets
Aug 09 22:59:03 dist-test-slave-vxj2 krb5kdc[28689](info): commencing operation
krb5kdc: starting...
W20260809 22:59:04.116037 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:59:04 dist-test-slave-vxj2 krb5kdc[28689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:59:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:59:04.335037 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:43459
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:43459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:04.661846 28705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:04.662187 28705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:04.662351 28705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:04.669474 28705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:04.669616 28705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:04.669719 28705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:04.669833 28705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:04.669922 28705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:04.680716 28705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:43459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:43459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:04.682088 28705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:04.683390 28705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:04.695190 28710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:04.695160 28711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:04.695109 28713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:04.696152 28705 server_base.cc:1034] running on GCE node
I20260809 22:59:04.696997 28705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:04.698350 28705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:04.699550 28705 hybrid_clock.cc:648] HybridClock initialized: now 1786316344699494 us; error 47 us; skew 500 ppm
I20260809 22:59:04.699900 28705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:04 dist-test-slave-vxj2 krb5kdc[28689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316344, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:04.709403 28705 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:59:04.710961 28705 webserver.cc:460] Webserver started at http://127.24.31.254:33905/ using document root <none> and password file <none>
I20260809 22:59:04.711650 28705 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:04.711752 28705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:04.712028 28705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:04.715193 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60"
format_stamp: "Formatted at 2026-08-09 22:59:04 on dist-test-slave-vxj2"
server_key: "fd5d18049b8bc6b5d7ad3fffaa456240"
server_key_iv: "87399b471420846dae496bddced96a53"
server_key_version: "encryptionkey@0"
I20260809 22:59:04.715937 28705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60"
format_stamp: "Formatted at 2026-08-09 22:59:04 on dist-test-slave-vxj2"
server_key: "fd5d18049b8bc6b5d7ad3fffaa456240"
server_key_iv: "87399b471420846dae496bddced96a53"
server_key_version: "encryptionkey@0"
I20260809 22:59:04.721347 28705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:59:04.724831 28722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:04.726258 28705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:04.726444 28705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60"
format_stamp: "Formatted at 2026-08-09 22:59:04 on dist-test-slave-vxj2"
server_key: "fd5d18049b8bc6b5d7ad3fffaa456240"
server_key_iv: "87399b471420846dae496bddced96a53"
server_key_version: "encryptionkey@0"
I20260809 22:59:04.726662 28705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:04.758286 28705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:04.762539 28705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:04.762840 28705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:04.781287 28705 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:43459
I20260809 22:59:04.781296 28776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:43459 every 8 connection(s)
I20260809 22:59:04.782660 28705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:04.786018 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 28705
I20260809 22:59:04.786283 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:04.786653 28777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:04.787169 24703 external_mini_cluster.cc:1428] Setting key d777322eb1a1ec9ffd8715d5806f486a
I20260809 22:59:04.801628 28777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60: Bootstrap starting.
Aug 09 22:59:04 dist-test-slave-vxj2 krb5kdc[28689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:04.805047 28777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:04.806455 28777 log.cc:824] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:04.809979 28777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60: No bootstrap required, opened a new log
I20260809 22:59:04.811983 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:04.789370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.24.31.254:43459)
0809 22:59:04.790289 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:04.790304 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:04.790348 (+ 44us) server_negotiation.cc:407] Connection header received
0809 22:59:04.791974 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:04.791995 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:04.792358 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:04.792675 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:04.794527 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:04.795833 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:04.797685 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:04.798368 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:04.798500 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:04.803400 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:04.803408 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:04.803425 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:04.803461 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:04.807323 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:04.808158 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:04.808165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:04.808173 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:04.808369 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:04.808721 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:04.808727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:04.808733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:04.809499 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:04.809798 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:04.810208 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 22:59:04.810680 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":464,"threads_started":1}
I20260809 22:59:04.818485 28777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43459 } }
I20260809 22:59:04.818810 28777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:04.818881 28777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce3a7f1cf5e4d80b1356c490ec05a60, State: Initialized, Role: FOLLOWER
I20260809 22:59:04.819412 28777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43459 } }
I20260809 22:59:04.819623 28777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:04.819720 28777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:04.819836 28777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:04.822070 28777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43459 } }
I20260809 22:59:04.822557 28777 leader_election.cc:302] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ce3a7f1cf5e4d80b1356c490ec05a60; no voters:
I20260809 22:59:04.823176 28777 leader_election.cc:288] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:04.823398 28783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:04.824622 28783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [term 1 LEADER]: Becoming Leader. State: Replica: 7ce3a7f1cf5e4d80b1356c490ec05a60, State: Running, Role: LEADER
I20260809 22:59:04.825139 28783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43459 } }
I20260809 22:59:04.825364 28777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:04.827785 28785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ce3a7f1cf5e4d80b1356c490ec05a60. Latest consensus state: current_term: 1 leader_uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43459 } } }
I20260809 22:59:04.828104 28785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:04.828735 28784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce3a7f1cf5e4d80b1356c490ec05a60" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43459 } } }
I20260809 22:59:04.829015 28784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:04.829195 28790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:04.835235 28790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:04.844587 28790 catalog_manager.cc:1347] Generated new cluster ID: 91a19815d5ba4e57b65474c73f0c5c52
I20260809 22:59:04.844697 28790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:04.878685 28790 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:04.880043 28790 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:04.890669 28790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ce3a7f1cf5e4d80b1356c490ec05a60: Generated new TSK 0
I20260809 22:59:04.891489 28790 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:59:04.947400 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:43459
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:05.278316 28807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:05.278618 28807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:05.278800 28807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:05.286135 28807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:05.286250 28807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:05.286427 28807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:05.296515 28807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:43459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:05.297808 28807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:05.299175 28807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:05.312448 28812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:05.312446 28815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:05.313241 28813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:05.313401 28807 server_base.cc:1034] running on GCE node
I20260809 22:59:05.314054 28807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:05.314970 28807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:05.316166 28807 hybrid_clock.cc:648] HybridClock initialized: now 1786316345316122 us; error 38 us; skew 500 ppm
I20260809 22:59:05.316480 28807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:05 dist-test-slave-vxj2 krb5kdc[28689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316345, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:05.319741 28807 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:59:05.321374 28807 webserver.cc:460] Webserver started at http://127.24.31.193:42847/ using document root <none> and password file <none>
I20260809 22:59:05.322068 28807 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:05.322208 28807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:05.322528 28807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:05.325532 28807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "7053ea3c43804e8eb60a1b66b354c8c0"
format_stamp: "Formatted at 2026-08-09 22:59:05 on dist-test-slave-vxj2"
server_key: "0d1f3e81612dd8bad5954d61c42c27b5"
server_key_iv: "c7e24f420ade14d1acbb23d322d0bd3c"
server_key_version: "encryptionkey@0"
I20260809 22:59:05.326462 28807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "7053ea3c43804e8eb60a1b66b354c8c0"
format_stamp: "Formatted at 2026-08-09 22:59:05 on dist-test-slave-vxj2"
server_key: "0d1f3e81612dd8bad5954d61c42c27b5"
server_key_iv: "c7e24f420ade14d1acbb23d322d0bd3c"
server_key_version: "encryptionkey@0"
I20260809 22:59:05.331836 28807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:59:05.335306 28826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:05.336593 28807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:59:05.336760 28807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "7053ea3c43804e8eb60a1b66b354c8c0"
format_stamp: "Formatted at 2026-08-09 22:59:05 on dist-test-slave-vxj2"
server_key: "0d1f3e81612dd8bad5954d61c42c27b5"
server_key_iv: "c7e24f420ade14d1acbb23d322d0bd3c"
server_key_version: "encryptionkey@0"
I20260809 22:59:05.336987 28807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:05.351318 28807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:05.355634 28807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:05.355952 28807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:05.357188 28807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:05.358700 28807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:05.358783 28807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:05.358911 28807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:05.358978 28807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:05.398257 28807 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:34427
I20260809 22:59:05.398285 28951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:34427 every 8 connection(s)
I20260809 22:59:05.399621 28807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:05.409240 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 28807
I20260809 22:59:05.409476 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:59:05.410326 24703 external_mini_cluster.cc:1428] Setting key 273514ab4b07f290ffbf674bee060d9f
Aug 09 22:59:05 dist-test-slave-vxj2 krb5kdc[28689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316345, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:05.421586 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:05.402292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:36303 (local address 127.24.31.254:43459)
0809 22:59:05.402780 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:05.402784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:05.403519 (+ 735us) server_negotiation.cc:407] Connection header received
0809 22:59:05.405415 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:05.405421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:05.405508 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:05.405645 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:05.407151 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:05.407893 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:05.409654 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:05.410348 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:05.410511 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:05.414442 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:05.414452 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:05.414460 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:05.414505 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:05.418160 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:05.418880 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:05.418888 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:05.418894 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:05.419058 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:05.419477 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:05.419485 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:05.419490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:05.419892 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:05.420064 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:05.420959 (+ 895us) server_negotiation.cc:299] Negotiation successful
0809 22:59:05.421290 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":276,"threads_started":1}
I20260809 22:59:05.422981 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:05.402626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:43459 (local address 127.24.31.193:36303)
0809 22:59:05.403333 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:05.403380 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:05.405081 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:05.405888 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:05.405896 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:05.406395 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:05.406973 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:05.406990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:05.408078 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:05.408082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:05.409460 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:05.409473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:05.410593 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:05.410598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:05.410799 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:05.411614 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:05.411643 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:05.414212 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:05.418312 (+ 4100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:05.418318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:05.418330 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:05.418743 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:05.419166 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:05.419170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:05.419172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:05.419357 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:05.420055 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:05.420067 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:05.420530 (+ 463us) client_negotiation.cc:769] Sending connection context
0809 22:59:05.420850 (+ 320us) client_negotiation.cc:240] Negotiation successful
0809 22:59:05.421501 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":279,"threads_started":1}
I20260809 22:59:05.424793 28953 heartbeater.cc:347] Connected to a master server at 127.24.31.254:43459
I20260809 22:59:05.425117 28953 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:05.425932 28953 heartbeater.cc:511] Master 127.24.31.254:43459 requested a full tablet report, sending...
I20260809 22:59:05.428282 28740 ts_manager.cc:194] Registered new tserver with Master: 7053ea3c43804e8eb60a1b66b354c8c0 (127.24.31.193:34427)
I20260809 22:59:05.430678 28740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:36303
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:59:05.460326 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:43459
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:05.784224 28959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:05.784652 28959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:05.784840 28959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:05.792382 28959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:05.792536 28959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:05.792724 28959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:05.802646 28959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:43459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:05.803934 28959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:05.805318 28959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:05.818828 28964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:05.818928 28967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:05.818899 28965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:05.819718 28959 server_base.cc:1034] running on GCE node
I20260809 22:59:05.820319 28959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:05.821312 28959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:05.822536 28959 hybrid_clock.cc:648] HybridClock initialized: now 1786316345822478 us; error 51 us; skew 500 ppm
I20260809 22:59:05.822854 28959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:05 dist-test-slave-vxj2 krb5kdc[28689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316345, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:05.826103 28959 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:59:05.827663 28959 webserver.cc:460] Webserver started at http://127.24.31.194:37091/ using document root <none> and password file <none>
I20260809 22:59:05.828364 28959 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:05.828517 28959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:05.828833 28959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:05.831995 28959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "26f97ea298ba4c47bc8fe9ac27131110"
format_stamp: "Formatted at 2026-08-09 22:59:05 on dist-test-slave-vxj2"
server_key: "a7c3a6b2db29beaf5c70fcdb9ee11f5b"
server_key_iv: "9192710f22f3e0e8cf5ce96407a43d5d"
server_key_version: "encryptionkey@0"
I20260809 22:59:05.832803 28959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "26f97ea298ba4c47bc8fe9ac27131110"
format_stamp: "Formatted at 2026-08-09 22:59:05 on dist-test-slave-vxj2"
server_key: "a7c3a6b2db29beaf5c70fcdb9ee11f5b"
server_key_iv: "9192710f22f3e0e8cf5ce96407a43d5d"
server_key_version: "encryptionkey@0"
I20260809 22:59:05.838330 28959 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 22:59:05.841904 28974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:05.843194 28959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:59:05.843372 28959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "26f97ea298ba4c47bc8fe9ac27131110"
format_stamp: "Formatted at 2026-08-09 22:59:05 on dist-test-slave-vxj2"
server_key: "a7c3a6b2db29beaf5c70fcdb9ee11f5b"
server_key_iv: "9192710f22f3e0e8cf5ce96407a43d5d"
server_key_version: "encryptionkey@0"
I20260809 22:59:05.843585 28959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:05.859037 28959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:05.863091 28959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:05.863428 28959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:05.864439 28959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:05.865851 28959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:05.865967 28959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:05.866138 28959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:05.866222 28959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:05.900250 28959 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:45031
I20260809 22:59:05.900292 29105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:45031 every 8 connection(s)
I20260809 22:59:05.901718 28959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:05.909575 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 28959
I20260809 22:59:05.909778 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:59:05.910271 24703 external_mini_cluster.cc:1428] Setting key 8de98c98f1039485765ad6f1b4cb3571
Aug 09 22:59:05 dist-test-slave-vxj2 krb5kdc[28689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316345, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:05.922780 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:05.904283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:56131 (local address 127.24.31.254:43459)
0809 22:59:05.904482 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:05.904487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:05.905619 (+ 1132us) server_negotiation.cc:407] Connection header received
0809 22:59:05.907556 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:05.907561 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:05.907633 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:05.907721 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:05.909221 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:05.910046 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:05.912000 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:05.912616 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:05.912716 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:05.916982 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:05.916987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:05.916991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:05.917024 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:05.919650 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:05.920404 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:05.920416 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:05.920419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:05.920548 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:05.920994 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:05.920999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:05.921004 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:05.921336 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:05.921511 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:05.922204 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 22:59:05.922435 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:59:05.923926 29108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:05.904712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:43459 (local address 127.24.31.194:56131)
0809 22:59:05.905457 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:05.905501 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:05.907224 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:05.907936 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:05.907945 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:05.908443 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:05.909008 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:05.909025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:05.910206 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:05.910212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:05.911844 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:05.911857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:05.912853 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:05.912861 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:05.913239 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:05.913959 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:05.914004 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:05.916744 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:05.919801 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:05.919810 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:05.919825 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:05.920271 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:05.920672 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:05.920676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:05.920680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:05.920873 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:05.921474 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:05.921481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:05.921822 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 22:59:05.922097 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 22:59:05.922645 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":271,"threads_started":1}
I20260809 22:59:05.925485 29106 heartbeater.cc:347] Connected to a master server at 127.24.31.254:43459
I20260809 22:59:05.925836 29106 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:05.926735 29106 heartbeater.cc:511] Master 127.24.31.254:43459 requested a full tablet report, sending...
I20260809 22:59:05.928393 28740 ts_manager.cc:194] Registered new tserver with Master: 26f97ea298ba4c47bc8fe9ac27131110 (127.24.31.194:45031)
I20260809 22:59:05.929288 28740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:56131
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:59:05.960161 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:43459
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:45883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:06.287815 29111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:06.288066 29111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:06.288173 29111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:06.295176 29111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:06.295281 29111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:06.295392 29111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:06.304695 29111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:45883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:43459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:06.305924 29111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:06.307276 29111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:06.320493 29117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:06.320557 29121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:06.320561 29118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:06.321627 29111 server_base.cc:1034] running on GCE node
I20260809 22:59:06.322240 29111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:06.323295 29111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:06.324507 29111 hybrid_clock.cc:648] HybridClock initialized: now 1786316346324453 us; error 46 us; skew 500 ppm
I20260809 22:59:06.324801 29111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:06 dist-test-slave-vxj2 krb5kdc[28689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316346, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:06.328140 29111 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:59:06.329560 29111 webserver.cc:460] Webserver started at http://127.24.31.195:39943/ using document root <none> and password file <none>
I20260809 22:59:06.330314 29111 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:06.330420 29111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:06.330681 29111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:06.333648 29111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "fc7d31d281a541488606728645b94ef7"
format_stamp: "Formatted at 2026-08-09 22:59:06 on dist-test-slave-vxj2"
server_key: "b683bdd61394511477b51cf41d8a6d7b"
server_key_iv: "5e65f682bb937b9e4aae79194364c7e0"
server_key_version: "encryptionkey@0"
I20260809 22:59:06.334548 29111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "fc7d31d281a541488606728645b94ef7"
format_stamp: "Formatted at 2026-08-09 22:59:06 on dist-test-slave-vxj2"
server_key: "b683bdd61394511477b51cf41d8a6d7b"
server_key_iv: "5e65f682bb937b9e4aae79194364c7e0"
server_key_version: "encryptionkey@0"
I20260809 22:59:06.339735 29111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:59:06.343425 29128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:06.344727 29111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:59:06.344918 29111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "fc7d31d281a541488606728645b94ef7"
format_stamp: "Formatted at 2026-08-09 22:59:06 on dist-test-slave-vxj2"
server_key: "b683bdd61394511477b51cf41d8a6d7b"
server_key_iv: "5e65f682bb937b9e4aae79194364c7e0"
server_key_version: "encryptionkey@0"
I20260809 22:59:06.345127 29111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:06.367169 29111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:06.371274 29111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:06.371510 29111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:06.372424 29111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:06.373742 29111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:06.373821 29111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:06.373903 29111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:06.373956 29111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:06.408027 29111 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:42847
I20260809 22:59:06.408080 29253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:42847 every 8 connection(s)
I20260809 22:59:06.409276 29111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:06.410902 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 29111
I20260809 22:59:06.411110 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:59:06.411561 24703 external_mini_cluster.cc:1428] Setting key 9ca997fc39be7b3e5d9f36de37a04751
Aug 09 22:59:06 dist-test-slave-vxj2 krb5kdc[28689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316346, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:06.430055 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:06.411893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:41997 (local address 127.24.31.254:43459)
0809 22:59:06.412122 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:06.412129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:06.413357 (+ 1228us) server_negotiation.cc:407] Connection header received
0809 22:59:06.415708 (+ 2351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:06.415713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:06.415800 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:06.415927 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:06.417381 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.418152 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.419654 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.420092 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.420230 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:06.423599 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:06.423605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:06.423610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:06.423652 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:06.426951 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.427673 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.427678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.427688 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.427815 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.428261 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.428270 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.428272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.428573 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:06.428738 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:06.429486 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 22:59:06.429701 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:59:06.431043 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:06.412321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:43459 (local address 127.24.31.195:41997)
0809 22:59:06.413183 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:06.413251 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:06.415455 (+ 2204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:06.416149 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:06.416156 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:06.416631 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:06.417201 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:06.417238 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.418299 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.418305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:06.419512 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:06.419523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.420312 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.420322 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:06.420575 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:06.421213 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:06.421238 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:06.423400 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:06.427088 (+ 3688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:06.427096 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:06.427111 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:06.427542 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:06.427939 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:06.427944 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:06.427948 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:06.428136 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:06.428711 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:06.428718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:06.429051 (+ 333us) client_negotiation.cc:769] Sending connection context
0809 22:59:06.429376 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 22:59:06.429879 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":308,"threads_started":1}
I20260809 22:59:06.432613 29254 heartbeater.cc:347] Connected to a master server at 127.24.31.254:43459
I20260809 22:59:06.432924 29254 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:06.433941 29254 heartbeater.cc:511] Master 127.24.31.254:43459 requested a full tablet report, sending...
I20260809 22:59:06.434000 28953 heartbeater.cc:503] Master 127.24.31.254:43459 was elected leader, sending a full tablet report...
I20260809 22:59:06.435657 28739 ts_manager.cc:194] Registered new tserver with Master: fc7d31d281a541488606728645b94ef7 (127.24.31.195:42847)
I20260809 22:59:06.436481 28739 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:41997
I20260809 22:59:06.439244 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:59:06 dist-test-slave-vxj2 krb5kdc[28689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:59:06 dist-test-slave-vxj2 krb5kdc[28689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:59:06.490080 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:06.475464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.24.31.193:34427)
0809 22:59:06.476113 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:06.476119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:06.476147 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:59:06.476208 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:06.476214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:06.476427 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:06.476557 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:06.478012 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.479083 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.480266 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.480876 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.480971 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:06.484748 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:06.484767 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:06.484780 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:06.484812 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:06.487781 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.488337 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.488343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.488345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.488461 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.488808 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.488811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.488813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.489122 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:06.489404 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:06.489583 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 22:59:06.489796 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":323,"threads_started":1}
Aug 09 22:59:06 dist-test-slave-vxj2 krb5kdc[28689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:06.507382 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:06.494619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.24.31.254:43459)
0809 22:59:06.494823 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:06.494829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:06.494943 (+ 114us) server_negotiation.cc:407] Connection header received
0809 22:59:06.495060 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:06.495064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:06.495122 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:06.495200 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:06.496552 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.497342 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.498500 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.498897 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.498987 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:06.502715 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:06.502719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:06.502723 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:06.502755 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:06.505104 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.505705 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.505709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.505711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.505834 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.506143 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.506146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.506148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.506436 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:06.506646 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:06.506886 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 22:59:06.507038 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 22:59:06.508128 28739 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41068
I20260809 22:59:06.509336 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28807
I20260809 22:59:06.522266 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:06.510373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60226 (local address 127.24.31.193:34427)
0809 22:59:06.510543 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:06.510547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:06.510712 (+ 165us) server_negotiation.cc:407] Connection header received
0809 22:59:06.510836 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:06.510840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:06.510908 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:06.510999 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:06.512123 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.512817 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.514071 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.514812 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.514969 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:06.517212 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:06.517226 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:06.517231 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:06.517278 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:06.519866 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.520446 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.520450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.520452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.520601 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.520960 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.520964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.520967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.521330 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:06.521529 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:06.521839 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 22:59:06.522019 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 22:59:06.522897 28947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:06.615674 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 28807
I20260809 22:59:06.635221 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28959
Aug 09 22:59:06 dist-test-slave-vxj2 krb5kdc[28689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 22:59:06.652004 29267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:06.636218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51002 (local address 127.24.31.194:45031)
0809 22:59:06.636781 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:06.636789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:06.636817 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:59:06.636885 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:06.636892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:06.637114 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:06.637317 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:06.638462 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.639552 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.640809 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.641539 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.641669 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:06.645941 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:06.645953 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:06.645968 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:06.646002 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:06.649080 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.649784 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.649791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.649793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.649922 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.650343 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.650349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.650352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.650694 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:06.651011 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:06.651364 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 22:59:06.651622 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":271,"threads_started":1}
I20260809 22:59:06.653344 29101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:06.736513 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 28959
I20260809 22:59:06.755378 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 29111
Aug 09 22:59:06 dist-test-slave-vxj2 krb5kdc[28689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 22:59:06.772915 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:06.756426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.24.31.195:42847)
0809 22:59:06.757242 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:06.757249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:06.757279 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:59:06.757341 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:06.757348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:06.757572 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:06.757699 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:06.759102 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.760459 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.762228 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:06.762803 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:06.762901 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:06.767415 (+ 4514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:06.767426 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:06.767453 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:06.767493 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:06.770214 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.770871 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.770878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.770880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.771001 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:06.771409 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:06.771412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:06.771414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:06.771746 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:06.771970 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:06.772274 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 22:59:06.772497 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":430,"threads_started":1}
I20260809 22:59:06.774273 29249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:06.864008 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 29111
I20260809 22:59:06.882537 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 28705
I20260809 22:59:06.883755 28772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:06.991518 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 28705
2026-08-09T22:59:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4039 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:59:07 dist-test-slave-vxj2 krb5kdc[29275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:59:07 dist-test-slave-vxj2 krb5kdc[29275](info): set up 2 sockets
Aug 09 22:59:07 dist-test-slave-vxj2 krb5kdc[29275](info): commencing operation
krb5kdc: starting...
W20260809 22:59:08.150739 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.008s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:59:08 dist-test-slave-vxj2 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:59:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:59:08.368614 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:33765
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:33765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:08.697003 29288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:08.697307 29288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:08.697504 29288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:08.704568 29288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:08.704674 29288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:08.704767 29288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:08.704882 29288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:08.704977 29288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:08.714782 29288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:33765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:33765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:08.716144 29288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:08.717504 29288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:08.728861 29293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:08.729015 29295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:08.729143 29297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:08.730167 29288 server_base.cc:1034] running on GCE node
I20260809 22:59:08.731007 29288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:08.732322 29288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:08.733520 29288 hybrid_clock.cc:648] HybridClock initialized: now 1786316348733479 us; error 37 us; skew 500 ppm
I20260809 22:59:08.733871 29288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:08 dist-test-slave-vxj2 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316348, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:08.736969 29288 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:59:08.738234 29288 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:08.738369 29288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:08.738690 29288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:08.741611 29288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "7887dbf678fe47fea244ded8d3374eb1"
format_stamp: "Formatted at 2026-08-09 22:59:08 on dist-test-slave-vxj2"
server_key: "4c4421765cff41d3500c1c437e6a38f8"
server_key_iv: "edc08eec15dd349f884b35a4a3782404"
server_key_version: "encryptionkey@0"
I20260809 22:59:08.742411 29288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "7887dbf678fe47fea244ded8d3374eb1"
format_stamp: "Formatted at 2026-08-09 22:59:08 on dist-test-slave-vxj2"
server_key: "4c4421765cff41d3500c1c437e6a38f8"
server_key_iv: "edc08eec15dd349f884b35a4a3782404"
server_key_version: "encryptionkey@0"
I20260809 22:59:08.747349 29288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:59:08.750803 29303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:08.752099 29288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:59:08.752283 29288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "7887dbf678fe47fea244ded8d3374eb1"
format_stamp: "Formatted at 2026-08-09 22:59:08 on dist-test-slave-vxj2"
server_key: "4c4421765cff41d3500c1c437e6a38f8"
server_key_iv: "edc08eec15dd349f884b35a4a3782404"
server_key_version: "encryptionkey@0"
I20260809 22:59:08.752512 29288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:08.765004 29288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:08.769356 29288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:08.769631 29288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:08.788566 29288 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:33765
I20260809 22:59:08.788586 29359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:33765 every 8 connection(s)
I20260809 22:59:08.789532 29288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:08.793210 29360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:08.799824 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 29288
I20260809 22:59:08.800074 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:08.800735 24703 external_mini_cluster.cc:1428] Setting key 666e0b5c76d56bf97a263669544012d2
I20260809 22:59:08.804673 29360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1: Bootstrap starting.
I20260809 22:59:08.808660 29360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:08.810161 29360 log.cc:824] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:08.813103 29360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1: No bootstrap required, opened a new log
Aug 09 22:59:08 dist-test-slave-vxj2 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:08.821040 29360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7887dbf678fe47fea244ded8d3374eb1" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33765 } }
I20260809 22:59:08.821501 29360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:08.821626 29360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7887dbf678fe47fea244ded8d3374eb1, State: Initialized, Role: FOLLOWER
I20260809 22:59:08.822196 29360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7887dbf678fe47fea244ded8d3374eb1" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33765 } }
I20260809 22:59:08.822440 29360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:08.822590 29360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:08.822790 29360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:08.824491 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:08.802925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55168 (local address 127.24.31.254:33765)
0809 22:59:08.803795 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:08.803811 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:08.803862 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:59:08.806388 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:08.806420 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:08.807021 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:08.807477 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:08.808916 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:08.810394 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:08.811698 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:08.812366 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:08.812581 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:08.816743 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:08.816755 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:08.816769 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:08.816813 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:08.820040 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:08.820731 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:08.820741 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:08.820755 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:08.820926 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:08.821408 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:08.821413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:08.821418 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:08.822209 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:08.822526 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:08.822990 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 22:59:08.823441 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":344,"threads_started":1}
I20260809 22:59:08.825414 29360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7887dbf678fe47fea244ded8d3374eb1" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33765 } }
I20260809 22:59:08.825907 29360 leader_election.cc:302] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7887dbf678fe47fea244ded8d3374eb1; no voters:
I20260809 22:59:08.826622 29360 leader_election.cc:288] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:08.826735 29366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:08.829275 29360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:08.829457 29366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [term 1 LEADER]: Becoming Leader. State: Replica: 7887dbf678fe47fea244ded8d3374eb1, State: Running, Role: LEADER
I20260809 22:59:08.830009 29366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7887dbf678fe47fea244ded8d3374eb1" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33765 } }
I20260809 22:59:08.833335 29369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7887dbf678fe47fea244ded8d3374eb1. Latest consensus state: current_term: 1 leader_uuid: "7887dbf678fe47fea244ded8d3374eb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7887dbf678fe47fea244ded8d3374eb1" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33765 } } }
I20260809 22:59:08.833758 29369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:08.834327 29368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7887dbf678fe47fea244ded8d3374eb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7887dbf678fe47fea244ded8d3374eb1" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33765 } } }
I20260809 22:59:08.834733 29368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:08.841650 29377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:08.846028 29377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:08.854303 29377 catalog_manager.cc:1347] Generated new cluster ID: ac04e55e2631469c90cfe015e0c843bc
I20260809 22:59:08.854419 29377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:08.874344 29377 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:08.876207 29377 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:08.890347 29377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1: Generated new TSK 0
I20260809 22:59:08.891438 29377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:59:08.953939 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33765
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:09.288508 29389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:09.288884 29389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:09.289103 29389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:09.296499 29389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:09.296660 29389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:09.296854 29389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:09.306818 29389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:09.308216 29389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:09.309580 29389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:09.322655 29396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:09.322696 29395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:09.322736 29398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:09.323439 29389 server_base.cc:1034] running on GCE node
I20260809 22:59:09.324056 29389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:09.324995 29389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:09.326227 29389 hybrid_clock.cc:648] HybridClock initialized: now 1786316349326186 us; error 68 us; skew 500 ppm
I20260809 22:59:09.326577 29389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:09 dist-test-slave-vxj2 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316349, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:09.330089 29389 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:59:09.331353 29389 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:09.331492 29389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:09.331845 29389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:09.334914 29389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "6b9f73d0deda479cb610d2605aa8f4fb"
format_stamp: "Formatted at 2026-08-09 22:59:09 on dist-test-slave-vxj2"
server_key: "19b11e962419437c758e25408553cee2"
server_key_iv: "724d0f16e464bfa0856109193334e957"
server_key_version: "encryptionkey@0"
I20260809 22:59:09.335738 29389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "6b9f73d0deda479cb610d2605aa8f4fb"
format_stamp: "Formatted at 2026-08-09 22:59:09 on dist-test-slave-vxj2"
server_key: "19b11e962419437c758e25408553cee2"
server_key_iv: "724d0f16e464bfa0856109193334e957"
server_key_version: "encryptionkey@0"
I20260809 22:59:09.341046 29389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:59:09.344576 29406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:09.345994 29389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:09.346184 29389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "6b9f73d0deda479cb610d2605aa8f4fb"
format_stamp: "Formatted at 2026-08-09 22:59:09 on dist-test-slave-vxj2"
server_key: "19b11e962419437c758e25408553cee2"
server_key_iv: "724d0f16e464bfa0856109193334e957"
server_key_version: "encryptionkey@0"
I20260809 22:59:09.346423 29389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:09.374049 29389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:09.378563 29389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:09.378861 29389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:09.379843 29389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:09.382525 29389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:09.382648 29389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:09.382787 29389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:09.382887 29389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:09.424983 29389 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:35627
I20260809 22:59:09.425076 29532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:35627 every 8 connection(s)
I20260809 22:59:09.425906 29389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:09.427194 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 29389
I20260809 22:59:09.427438 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:59:09.428287 24703 external_mini_cluster.cc:1428] Setting key 339b34bc0e3369565fa40f6aaf79e4c8
Aug 09 22:59:09 dist-test-slave-vxj2 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316349, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:09.451663 29537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:09.428743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:42961 (local address 127.24.31.254:33765)
0809 22:59:09.429291 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:09.429295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:09.430488 (+ 1193us) server_negotiation.cc:407] Connection header received
0809 22:59:09.433538 (+ 3050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:09.433543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:09.433619 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:09.433728 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:09.435960 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:09.437120 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:09.439496 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:09.439940 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:09.440082 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:09.444370 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:09.444377 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:09.444381 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:09.444425 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:09.447644 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:09.448710 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:09.448714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:09.448716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:09.448829 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:09.449602 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:09.449605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:09.449607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:09.449868 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:09.450047 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:09.451127 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0809 22:59:09.451367 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":285,"threads_started":1}
I20260809 22:59:09.453342 29538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:09.429373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:33765 (local address 127.24.31.193:42961)
0809 22:59:09.430337 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:09.430377 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:09.433186 (+ 2809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:09.434025 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:09.434040 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:09.434870 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:09.435735 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:09.435764 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:09.437348 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:09.437356 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:09.439312 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:09.439325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:09.440199 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:09.440207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:09.440429 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:09.441243 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:09.441272 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:09.444115 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:09.447801 (+ 3686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:09.447814 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:09.447834 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:09.448595 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:09.448969 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:09.448977 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:09.448984 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:09.449293 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:09.450006 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:09.450015 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:09.450565 (+ 550us) client_negotiation.cc:769] Sending connection context
0809 22:59:09.450973 (+ 408us) client_negotiation.cc:240] Negotiation successful
0809 22:59:09.451673 (+ 700us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":374,"threads_started":1}
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
I20260809 22:59:09.455071 29533 heartbeater.cc:347] Connected to a master server at 127.24.31.254:33765
I20260809 22:59:09.455369 29533 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:09.456135 29533 heartbeater.cc:511] Master 127.24.31.254:33765 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
I20260809 22:59:09.458490 29320 ts_manager.cc:194] Registered new tserver with Master: 6b9f73d0deda479cb610d2605aa8f4fb (127.24.31.193:35627)
I20260809 22:59:09.460657 29320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:42961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:59:09.485131 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33765
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:09.848336 29543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:09.848654 29543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:09.848838 29543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:09.856349 29543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:09.856498 29543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:09.856680 29543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:09.866791 29543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:09.868121 29543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:09.869460 29543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:09.882467 29550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:09.882483 29552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:09.882684 29549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:09.883360 29543 server_base.cc:1034] running on GCE node
I20260809 22:59:09.884064 29543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:09.885974 29543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:09.887200 29543 hybrid_clock.cc:648] HybridClock initialized: now 1786316349887144 us; error 45 us; skew 500 ppm
I20260809 22:59:09.887524 29543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:09 dist-test-slave-vxj2 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316349, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:09.893612 29543 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:59:09.894766 29543 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:09.894902 29543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:09.895226 29543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:09.898295 29543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "a5eebbf9c601413785ef7905ced3a385"
format_stamp: "Formatted at 2026-08-09 22:59:09 on dist-test-slave-vxj2"
server_key: "6928f6a559d6a93b133485de3d3b599a"
server_key_iv: "278159f51934ba8060e49cab547310af"
server_key_version: "encryptionkey@0"
I20260809 22:59:09.899137 29543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "a5eebbf9c601413785ef7905ced3a385"
format_stamp: "Formatted at 2026-08-09 22:59:09 on dist-test-slave-vxj2"
server_key: "6928f6a559d6a93b133485de3d3b599a"
server_key_iv: "278159f51934ba8060e49cab547310af"
server_key_version: "encryptionkey@0"
I20260809 22:59:09.904196 29543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:59:09.907765 29558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:09.909101 29543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 22:59:09.909296 29543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "a5eebbf9c601413785ef7905ced3a385"
format_stamp: "Formatted at 2026-08-09 22:59:09 on dist-test-slave-vxj2"
server_key: "6928f6a559d6a93b133485de3d3b599a"
server_key_iv: "278159f51934ba8060e49cab547310af"
server_key_version: "encryptionkey@0"
I20260809 22:59:09.909516 29543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:09.935572 29543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:09.940058 29543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:09.940295 29543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:09.941356 29543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:09.942705 29543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:09.942798 29543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:09.942870 29543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:09.942925 29543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:09.977589 29543 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:37483
I20260809 22:59:09.977705 29685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:37483 every 8 connection(s)
I20260809 22:59:09.978523 29543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:09.985960 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 29543
I20260809 22:59:09.986181 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:59:09.986774 24703 external_mini_cluster.cc:1428] Setting key 4302dc8f73fc8311391eaff4171173b0
Aug 09 22:59:09 dist-test-slave-vxj2 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316349, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:10.000341 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:09.981258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:34835 (local address 127.24.31.254:33765)
0809 22:59:09.981708 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:09.981712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:09.982644 (+ 932us) server_negotiation.cc:407] Connection header received
0809 22:59:09.984410 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:09.984414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:09.984483 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:09.984583 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:09.986172 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:09.987288 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:09.989447 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:09.990190 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:09.990343 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:09.994036 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:09.994045 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:09.994052 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:09.994096 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:09.997266 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:09.997995 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:09.998000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:09.998005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:09.998144 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:09.998589 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:09.998593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:09.998595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:09.998853 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:09.999019 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:09.999689 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 22:59:09.999922 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":220,"threads_started":1}
I20260809 22:59:10.001384 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:09.981801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:33765 (local address 127.24.31.194:34835)
0809 22:59:09.982474 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:09.982517 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:09.984151 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:09.984773 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:09.984781 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:09.985388 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:09.985937 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:09.985964 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:09.987445 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:09.987452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:09.989308 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:09.989322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:09.990398 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:09.990403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:09.990586 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:09.991286 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:09.991309 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:09.993767 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:09.997400 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:09.997410 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:09.997426 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:09.997865 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:09.998268 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:09.998273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:09.998277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:09.998461 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:09.998981 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:09.998989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:09.999349 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 22:59:09.999597 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 22:59:10.000145 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":284,"threads_started":1}
I20260809 22:59:10.003057 29686 heartbeater.cc:347] Connected to a master server at 127.24.31.254:33765
I20260809 22:59:10.003376 29686 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:10.004047 29686 heartbeater.cc:511] Master 127.24.31.254:33765 requested a full tablet report, sending...
I20260809 22:59:10.005877 29320 ts_manager.cc:194] Registered new tserver with Master: a5eebbf9c601413785ef7905ced3a385 (127.24.31.194:37483)
I20260809 22:59:10.007254 29320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:34835
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:59:10.040037 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33765
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:40391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:10.370724 29692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:10.371016 29692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:10.371181 29692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:10.378476 29692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:10.378587 29692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:10.378803 29692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:10.388646 29692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:40391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:10.390009 29692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:10.391286 29692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:10.404290 29697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:10.404248 29703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:10.404248 29701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:10.405100 29692 server_base.cc:1034] running on GCE node
I20260809 22:59:10.405778 29692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:10.406738 29692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:10.407999 29692 hybrid_clock.cc:648] HybridClock initialized: now 1786316350407954 us; error 57 us; skew 500 ppm
I20260809 22:59:10.408344 29692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:10 dist-test-slave-vxj2 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316350, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:10.411652 29692 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:59:10.412890 29692 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:10.413026 29692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:10.413482 29692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:10.416507 29692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "8e3756622d514263adb0f720980cf5f8"
format_stamp: "Formatted at 2026-08-09 22:59:10 on dist-test-slave-vxj2"
server_key: "05b80f5c85b505c4ee746369c5c12af8"
server_key_iv: "705f200683a3e16eff2359e8fc77af7e"
server_key_version: "encryptionkey@0"
I20260809 22:59:10.417418 29692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "8e3756622d514263adb0f720980cf5f8"
format_stamp: "Formatted at 2026-08-09 22:59:10 on dist-test-slave-vxj2"
server_key: "05b80f5c85b505c4ee746369c5c12af8"
server_key_iv: "705f200683a3e16eff2359e8fc77af7e"
server_key_version: "encryptionkey@0"
I20260809 22:59:10.422667 29692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:59:10.426716 29710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:10.428236 29692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:59:10.428421 29692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "8e3756622d514263adb0f720980cf5f8"
format_stamp: "Formatted at 2026-08-09 22:59:10 on dist-test-slave-vxj2"
server_key: "05b80f5c85b505c4ee746369c5c12af8"
server_key_iv: "705f200683a3e16eff2359e8fc77af7e"
server_key_version: "encryptionkey@0"
I20260809 22:59:10.428673 29692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:10.440927 29692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:10.445063 29692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:10.445365 29692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:10.446201 29692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:10.448108 29692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:10.448215 29692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:10.448323 29692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:10.448406 29692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:10.464425 29533 heartbeater.cc:503] Master 127.24.31.254:33765 was elected leader, sending a full tablet report...
I20260809 22:59:10.481815 29692 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:34289
I20260809 22:59:10.481921 29835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:34289 every 8 connection(s)
I20260809 22:59:10.482666 29692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:10.491917 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 29692
I20260809 22:59:10.492131 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:59:10.492714 24703 external_mini_cluster.cc:1428] Setting key 2f922576af9f2feec45e4943efeb00d2
Aug 09 22:59:10 dist-test-slave-vxj2 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316350, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:10.502667 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.485187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:34445 (local address 127.24.31.254:33765)
0809 22:59:10.485389 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:10.485394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:10.486554 (+ 1160us) server_negotiation.cc:407] Connection header received
0809 22:59:10.488305 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:10.488310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:10.488393 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:10.488496 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:10.489983 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.490735 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.492358 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.492982 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.493122 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.497264 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:10.497269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:10.497272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:10.497317 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:10.499833 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:10.500489 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:10.500493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:10.500495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:10.500610 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:10.501020 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:10.501024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:10.501026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:10.501419 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:10.501549 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:10.502255 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 22:59:10.502431 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 22:59:10.503540 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.485669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:33765 (local address 127.24.31.195:34445)
0809 22:59:10.486376 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:10.486414 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:10.488044 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:10.488688 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:10.488696 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:10.489264 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:10.489796 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.489830 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.490879 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.490883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.492198 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.492215 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.493235 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.493241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.493529 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.494315 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:10.494346 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:10.497030 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:10.499961 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:10.499967 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:10.499978 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:10.500376 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:10.500703 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:10.500706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:10.500708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:10.500875 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:10.501539 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:10.501544 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:10.501838 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 22:59:10.502061 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 22:59:10.502414 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":290,"threads_started":1}
I20260809 22:59:10.504981 29836 heartbeater.cc:347] Connected to a master server at 127.24.31.254:33765
I20260809 22:59:10.505312 29836 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:10.505900 29836 heartbeater.cc:511] Master 127.24.31.254:33765 requested a full tablet report, sending...
I20260809 22:59:10.507597 29319 ts_manager.cc:194] Registered new tserver with Master: 8e3756622d514263adb0f720980cf5f8 (127.24.31.195:34289)
I20260809 22:59:10.508658 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:59:10.508630 29319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:34445
I20260809 22:59:10.526659 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.514112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.24.31.254:33765)
0809 22:59:10.514314 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:10.514323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:10.514871 (+ 548us) server_negotiation.cc:407] Connection header received
0809 22:59:10.515038 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:10.515041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:10.515119 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:10.515205 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:10.516645 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.517474 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.518675 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.519126 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.519271 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.521607 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:10.521612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:10.521615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:10.521644 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:10.523911 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:10.524547 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:10.524554 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:10.524557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:10.524695 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:10.525072 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:10.525077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:10.525080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:10.525465 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:10.525740 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:10.526054 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 22:59:10.526282 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:59:10.530783 29319 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:59:10.538064 29319 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:59:10.583501 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.551265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.24.31.195:34289)
0809 22:59:10.573405 (+ 22140us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:10.573420 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:10.574460 (+ 1040us) server_negotiation.cc:407] Connection header received
0809 22:59:10.574585 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:10.574596 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:10.574935 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:10.575187 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:10.576456 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.578159 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.580721 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.582212 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.582372 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.582638 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:10.582978 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 22:59:10.583153 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21864,"thread_start_us":22192,"threads_started":1}
I20260809 22:59:10.583593 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.550612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:34289 (local address 127.0.0.1:51178)
0809 22:59:10.574134 (+ 23522us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:10.574156 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:10.574293 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:10.575647 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:10.575657 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:10.575709 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:10.576204 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.576219 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.578297 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.578301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.580551 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.580563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.582457 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.582467 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.582732 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.582772 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 22:59:10.582914 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 22:59:10.583111 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23391,"mutex_wait_us":1,"thread_start_us":25787,"threads_started":1}
I20260809 22:59:10.586136 29852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.550154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:35627 (local address 127.0.0.1:44894)
0809 22:59:10.561227 (+ 11073us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:10.561283 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:10.561534 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:10.578728 (+ 17194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:10.578733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:10.578758 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:10.579397 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.579412 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.581325 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.581331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.583868 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.583890 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.585226 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.585233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.585412 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.585452 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 22:59:10.585570 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 22:59:10.585697 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10658,"mutex_wait_us":129,"thread_start_us":26040,"threads_started":1}
I20260809 22:59:10.586125 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.550381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.24.31.193:35627)
0809 22:59:10.578174 (+ 27793us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:10.578181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:10.578209 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:59:10.578305 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:10.578313 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:10.578540 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:10.578732 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:10.579651 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.581128 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.584074 (+ 2946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.584992 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.585153 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.585412 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:10.585615 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 22:59:10.585771 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27613,"thread_start_us":27423,"threads_started":1}
I20260809 22:59:10.586905 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.550744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:37483 (local address 127.0.0.1:56768)
0809 22:59:10.575960 (+ 25216us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:10.575980 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:10.576096 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:10.578412 (+ 2316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:10.578419 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:10.578452 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:10.579012 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.579026 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.581046 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.581055 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.583823 (+ 2768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.583839 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.585973 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.585980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.586175 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.586205 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:59:10.586406 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 22:59:10.586547 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25048,"mutex_wait_us":187,"thread_start_us":9882,"threads_started":1}
I20260809 22:59:10.586995 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.551117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56768 (local address 127.24.31.194:37483)
0809 22:59:10.577056 (+ 25939us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:10.577066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:10.577096 (+ 30us) server_negotiation.cc:407] Connection header received
0809 22:59:10.577217 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:10.577224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:10.577494 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:10.578409 (+ 915us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:10.579250 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.580878 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.584036 (+ 3158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.585491 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.585630 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.585810 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:10.586483 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 22:59:10.586673 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25746,"thread_start_us":25613,"threads_started":1}
I20260809 22:59:10.588413 29761 tablet_service.cc:1467] Processing CreateTablet for tablet db6b19d4975840bfab8d6cf5f04e1d19 (DEFAULT_TABLE table=test-table [id=e7fe33b83abf418da0e6ac062a7b844b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:10.590039 29761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db6b19d4975840bfab8d6cf5f04e1d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:10.589915 29458 tablet_service.cc:1467] Processing CreateTablet for tablet db6b19d4975840bfab8d6cf5f04e1d19 (DEFAULT_TABLE table=test-table [id=e7fe33b83abf418da0e6ac062a7b844b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:10.590102 29613 tablet_service.cc:1467] Processing CreateTablet for tablet db6b19d4975840bfab8d6cf5f04e1d19 (DEFAULT_TABLE table=test-table [id=e7fe33b83abf418da0e6ac062a7b844b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:10.591339 29458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db6b19d4975840bfab8d6cf5f04e1d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:10.591355 29613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db6b19d4975840bfab8d6cf5f04e1d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:10.602553 29855 tablet_bootstrap.cc:492] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: Bootstrap starting.
I20260809 22:59:10.605825 29856 tablet_bootstrap.cc:492] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: Bootstrap starting.
I20260809 22:59:10.606458 29855 tablet_bootstrap.cc:654] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:10.607973 29855 log.cc:824] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:10.609508 29857 tablet_bootstrap.cc:492] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: Bootstrap starting.
I20260809 22:59:10.609612 29856 tablet_bootstrap.cc:654] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:10.610895 29856 log.cc:824] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:10.611559 29855 tablet_bootstrap.cc:492] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: No bootstrap required, opened a new log
I20260809 22:59:10.611863 29855 ts_tablet_manager.cc:1397] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260809 22:59:10.613197 29857 tablet_bootstrap.cc:654] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:10.614464 29857 log.cc:824] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:10.614540 29856 tablet_bootstrap.cc:492] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: No bootstrap required, opened a new log
I20260809 22:59:10.614794 29856 ts_tablet_manager.cc:1397] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:59:10.617656 29857 tablet_bootstrap.cc:492] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: No bootstrap required, opened a new log
I20260809 22:59:10.617930 29857 ts_tablet_manager.cc:1397] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 22:59:10.620234 29855 raft_consensus.cc:348] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } }
I20260809 22:59:10.620599 29855 raft_consensus.cc:374] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:10.620705 29855 raft_consensus.cc:729] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5eebbf9c601413785ef7905ced3a385, State: Initialized, Role: FOLLOWER
I20260809 22:59:10.621361 29855 consensus_queue.cc:260] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } }
I20260809 22:59:10.622464 29686 heartbeater.cc:503] Master 127.24.31.254:33765 was elected leader, sending a full tablet report...
I20260809 22:59:10.623018 29855 ts_tablet_manager.cc:1428] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 22:59:10.624699 29856 raft_consensus.cc:348] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } }
I20260809 22:59:10.624998 29856 raft_consensus.cc:374] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:10.625061 29856 raft_consensus.cc:729] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9f73d0deda479cb610d2605aa8f4fb, State: Initialized, Role: FOLLOWER
I20260809 22:59:10.625685 29856 consensus_queue.cc:260] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } }
I20260809 22:59:10.627378 29856 ts_tablet_manager.cc:1428] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260809 22:59:10.628628 29857 raft_consensus.cc:348] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } }
I20260809 22:59:10.629065 29857 raft_consensus.cc:374] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:10.629225 29857 raft_consensus.cc:729] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e3756622d514263adb0f720980cf5f8, State: Initialized, Role: FOLLOWER
I20260809 22:59:10.629837 29857 consensus_queue.cc:260] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } }
I20260809 22:59:10.631124 29836 heartbeater.cc:503] Master 127.24.31.254:33765 was elected leader, sending a full tablet report...
I20260809 22:59:10.631632 29857 ts_tablet_manager.cc:1428] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260809 22:59:10.678515 29535 tablet.cc:2369] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:59:10.730710 29687 tablet.cc:2369] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:59:10.734231 29837 tablet.cc:2369] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:59:10.799939 29861 raft_consensus.cc:482] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:10.800345 29861 raft_consensus.cc:504] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } }
I20260809 22:59:10.802237 29861 leader_election.cc:288] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b9f73d0deda479cb610d2605aa8f4fb (127.24.31.193:35627), 8e3756622d514263adb0f720980cf5f8 (127.24.31.195:34289)
I20260809 22:59:10.808993 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.802575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:50625 (local address 127.24.31.193:35627)
0809 22:59:10.802824 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:10.802834 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:10.803465 (+ 631us) server_negotiation.cc:407] Connection header received
0809 22:59:10.803734 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:10.803738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:10.803799 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:10.803901 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:10.804504 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.805336 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.807263 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.808008 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.808194 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.808274 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:10.808602 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 22:59:10.808725 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:59:10.809023 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.802787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:52375 (local address 127.24.31.195:34289)
0809 22:59:10.803051 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:10.803059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:10.803718 (+ 659us) server_negotiation.cc:407] Connection header received
0809 22:59:10.803922 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:10.803926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:10.803985 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:10.804085 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:10.804720 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.805490 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.807276 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.808051 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.808235 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.808324 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:10.808621 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 22:59:10.808747 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 22:59:10.809141 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.802493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:35627 (local address 127.24.31.194:50625)
0809 22:59:10.803336 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:10.803354 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:10.803560 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:10.803924 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:10.803928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:10.803953 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:10.804345 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.804356 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.805477 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.805482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.807105 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.807118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.808245 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.808252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.808434 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.808466 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:59:10.808574 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 22:59:10.808704 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":93,"thread_start_us":796,"threads_started":1}
I20260809 22:59:10.809602 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.802515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:34289 (local address 127.24.31.194:52375)
0809 22:59:10.803609 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:10.803623 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:10.803768 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:10.804102 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:10.804106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:10.804134 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:10.804557 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.804566 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.805627 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.805632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.807124 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:10.807137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.808282 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.808288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:10.808463 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.808486 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:59:10.808594 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 22:59:10.808713 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":34,"thread_start_us":510,"threads_started":1}
I20260809 22:59:10.810169 29478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db6b19d4975840bfab8d6cf5f04e1d19" candidate_uuid: "a5eebbf9c601413785ef7905ced3a385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" is_pre_election: true
I20260809 22:59:10.810479 29784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db6b19d4975840bfab8d6cf5f04e1d19" candidate_uuid: "a5eebbf9c601413785ef7905ced3a385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e3756622d514263adb0f720980cf5f8" is_pre_election: true
I20260809 22:59:10.810681 29478 raft_consensus.cc:2454] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5eebbf9c601413785ef7905ced3a385 in term 0.
I20260809 22:59:10.810948 29784 raft_consensus.cc:2454] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5eebbf9c601413785ef7905ced3a385 in term 0.
I20260809 22:59:10.811522 29559 leader_election.cc:302] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b9f73d0deda479cb610d2605aa8f4fb, a5eebbf9c601413785ef7905ced3a385; no voters:
I20260809 22:59:10.811964 29861 raft_consensus.cc:2781] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:10.812116 29861 raft_consensus.cc:482] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:10.812242 29861 raft_consensus.cc:3037] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:10.815022 29861 raft_consensus.cc:504] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } }
I20260809 22:59:10.815675 29861 leader_election.cc:288] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [CANDIDATE]: Term 1 election: Requested vote from peers 6b9f73d0deda479cb610d2605aa8f4fb (127.24.31.193:35627), 8e3756622d514263adb0f720980cf5f8 (127.24.31.195:34289)
I20260809 22:59:10.816066 29478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db6b19d4975840bfab8d6cf5f04e1d19" candidate_uuid: "a5eebbf9c601413785ef7905ced3a385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b9f73d0deda479cb610d2605aa8f4fb"
I20260809 22:59:10.816171 29784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db6b19d4975840bfab8d6cf5f04e1d19" candidate_uuid: "a5eebbf9c601413785ef7905ced3a385" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e3756622d514263adb0f720980cf5f8"
I20260809 22:59:10.816269 29478 raft_consensus.cc:3037] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:10.816417 29784 raft_consensus.cc:3037] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:10.818598 29478 raft_consensus.cc:2454] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5eebbf9c601413785ef7905ced3a385 in term 1.
I20260809 22:59:10.818738 29784 raft_consensus.cc:2454] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5eebbf9c601413785ef7905ced3a385 in term 1.
I20260809 22:59:10.819010 29559 leader_election.cc:302] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b9f73d0deda479cb610d2605aa8f4fb, a5eebbf9c601413785ef7905ced3a385; no voters:
I20260809 22:59:10.819272 29861 raft_consensus.cc:2781] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:10.819727 29861 raft_consensus.cc:686] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 1 LEADER]: Becoming Leader. State: Replica: a5eebbf9c601413785ef7905ced3a385, State: Running, Role: LEADER
I20260809 22:59:10.820236 29861 consensus_queue.cc:237] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } }
I20260809 22:59:10.824952 29319 catalog_manager.cc:5495] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5eebbf9c601413785ef7905ced3a385 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "a5eebbf9c601413785ef7905ced3a385" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5eebbf9c601413785ef7905ced3a385" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37483 } health_report { overall_health: HEALTHY } } }
I20260809 22:59:10.858722 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.853336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56782 (local address 127.24.31.194:37483)
0809 22:59:10.853564 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:10.853571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:10.853615 (+ 44us) server_negotiation.cc:407] Connection header received
0809 22:59:10.853875 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:10.853879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:10.853960 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:10.854114 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:10.854498 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.855203 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.856564 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.856978 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.857122 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.857648 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:10.858139 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:10.858312 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:10.858370 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 22:59:10.858460 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:59:10.866688 29478 raft_consensus.cc:1261] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [term 1 FOLLOWER]: Refusing update from remote peer a5eebbf9c601413785ef7905ced3a385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:10.866689 29784 raft_consensus.cc:1261] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [term 1 FOLLOWER]: Refusing update from remote peer a5eebbf9c601413785ef7905ced3a385: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:10.867654 29869 consensus_queue.cc:1035] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b9f73d0deda479cb610d2605aa8f4fb" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 35627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:10.868000 29861 consensus_queue.cc:1035] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e3756622d514263adb0f720980cf5f8" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:10.877781 29872 mvcc.cc:205] Tried to move back new op lower bound from 7316751773140279296 to 7316751772963405824. Current Snapshot: MvccSnapshot[applied={T|T < 7316751773140279296}]
I20260809 22:59:10.878688 29873 mvcc.cc:205] Tried to move back new op lower bound from 7316751773140279296 to 7316751772963405824. Current Snapshot: MvccSnapshot[applied={T|T < 7316751773140279296}]
I20260809 22:59:10.894040 29320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:10.894321 29320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:10.898110 29320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7887dbf678fe47fea244ded8d3374eb1: Sending DeleteTablet for 3 replicas of tablet db6b19d4975840bfab8d6cf5f04e1d19
I20260809 22:59:10.899225 29458 tablet_service.cc:1514] Processing DeleteTablet for tablet db6b19d4975840bfab8d6cf5f04e1d19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:10 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:44894
I20260809 22:59:10.899317 29761 tablet_service.cc:1514] Processing DeleteTablet for tablet db6b19d4975840bfab8d6cf5f04e1d19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:10 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:51178
I20260809 22:59:10.900419 29613 tablet_service.cc:1514] Processing DeleteTablet for tablet db6b19d4975840bfab8d6cf5f04e1d19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:10 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:56768
I20260809 22:59:10.902530 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 29389
I20260809 22:59:10.914578 29881 tablet_replica.cc:330] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: stopping tablet replica
I20260809 22:59:10.915174 29881 raft_consensus.cc:2229] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:10.915776 29881 raft_consensus.cc:2258] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:10.918720 29881 ts_tablet_manager.cc:1905] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:10.922255 29883 tablet_replica.cc:330] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: stopping tablet replica
I20260809 22:59:10.922662 29883 raft_consensus.cc:2229] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:59:10.923367 29883 raft_consensus.cc:2258] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:10.929128 29881 ts_tablet_manager.cc:1918] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:10.929358 29881 log.cc:1196] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/db6b19d4975840bfab8d6cf5f04e1d19
I20260809 22:59:10.929363 29883 ts_tablet_manager.cc:1905] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:10.929949 29881 ts_tablet_manager.cc:1939] T db6b19d4975840bfab8d6cf5f04e1d19 P 6b9f73d0deda479cb610d2605aa8f4fb: Deleting consensus metadata
Aug 09 22:59:10 dist-test-slave-vxj2 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:59:10.931869 29304 catalog_manager.cc:4841] TS 6b9f73d0deda479cb610d2605aa8f4fb (127.24.31.193:35627): tablet db6b19d4975840bfab8d6cf5f04e1d19 (table test-table [id=e7fe33b83abf418da0e6ac062a7b844b]) successfully deleted
I20260809 22:59:10.932643 29882 tablet_replica.cc:330] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: stopping tablet replica
I20260809 22:59:10.933264 29882 raft_consensus.cc:2229] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:10.933897 29882 raft_consensus.cc:2258] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:10.936168 29882 ts_tablet_manager.cc:1905] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:10.937980 29883 ts_tablet_manager.cc:1918] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:10.938163 29883 log.cc:1196] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/db6b19d4975840bfab8d6cf5f04e1d19
I20260809 22:59:10.938791 29883 ts_tablet_manager.cc:1939] T db6b19d4975840bfab8d6cf5f04e1d19 P a5eebbf9c601413785ef7905ced3a385: Deleting consensus metadata
I20260809 22:59:10.939355 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:10.904277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.24.31.193:35627)
0809 22:59:10.904525 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:10.904534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:10.916964 (+ 12430us) server_negotiation.cc:407] Connection header received
0809 22:59:10.917200 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:10.917205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:10.917298 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:10.917410 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:10.921713 (+ 4303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.922819 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.924881 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:10.925547 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:10.925650 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:10.931655 (+ 6005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:10.931672 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:10.931689 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:10.931735 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:10.935726 (+ 3991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:10.936641 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:10.936650 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:10.936655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:10.936826 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:10.937348 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:10.937353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:10.937357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:10.937787 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:10.938066 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:10.938691 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 22:59:10.938978 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 22:59:10.940538 29307 catalog_manager.cc:4841] TS a5eebbf9c601413785ef7905ced3a385 (127.24.31.194:37483): tablet db6b19d4975840bfab8d6cf5f04e1d19 (table test-table [id=e7fe33b83abf418da0e6ac062a7b844b]) successfully deleted
I20260809 22:59:10.940903 29528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:10.945595 29882 ts_tablet_manager.cc:1918] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:10.945783 29882 log.cc:1196] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/db6b19d4975840bfab8d6cf5f04e1d19
I20260809 22:59:10.946329 29882 ts_tablet_manager.cc:1939] T db6b19d4975840bfab8d6cf5f04e1d19 P 8e3756622d514263adb0f720980cf5f8: Deleting consensus metadata
I20260809 22:59:10.948292 29305 catalog_manager.cc:4841] TS 8e3756622d514263adb0f720980cf5f8 (127.24.31.195:34289): tablet db6b19d4975840bfab8d6cf5f04e1d19 (table test-table [id=e7fe33b83abf418da0e6ac062a7b844b]) successfully deleted
I20260809 22:59:11.050169 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 29389
I20260809 22:59:11.070580 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 29543
Aug 09 22:59:11 dist-test-slave-vxj2 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 22:59:11.089193 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:11.071427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56794 (local address 127.24.31.194:37483)
0809 22:59:11.071609 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:11.071613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:11.071635 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:59:11.071885 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:11.071889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:11.071953 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:11.072034 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:11.073660 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:11.074454 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:11.076215 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:11.076679 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:11.076769 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:11.082467 (+ 5698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:11.082483 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:11.082504 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:11.082549 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:11.086104 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:11.086961 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:11.086982 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:11.086984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:11.087136 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:11.087529 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:11.087544 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:11.087546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:11.087933 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:11.088290 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:11.088540 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 22:59:11.088792 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:59:11.089937 29681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:11.205669 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 29543
I20260809 22:59:11.228540 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 29692
Aug 09 22:59:11 dist-test-slave-vxj2 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 22:59:11.246097 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:11.229456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.24.31.195:34289)
0809 22:59:11.229663 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:11.229672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:11.229766 (+ 94us) server_negotiation.cc:407] Connection header received
0809 22:59:11.230003 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:11.230010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:11.230118 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:11.230234 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:11.231780 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:11.232599 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:11.234265 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:11.234679 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:11.234872 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:11.239771 (+ 4899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:11.239789 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:11.239806 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:11.239849 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:11.242996 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:11.243728 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:11.243736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:11.243739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:11.243902 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:11.244357 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:11.244362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:11.244364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:11.244741 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:11.245069 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:11.245472 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 22:59:11.245736 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:59:11.246851 29831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:11.336127 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 29692
I20260809 22:59:11.357373 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 29288
I20260809 22:59:11.358548 29354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:11.475519 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 29288
2026-08-09T22:59:11Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4485 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T22:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:11Z Disabled control of system clock
I20260809 22:59:11.548347 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:36101
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:36101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 22:59:11.894369 29895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:11.894677 29895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:11.894857 29895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:11.902105 29895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:11.902220 29895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:11.902308 29895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:11.902400 29895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:11.902484 29895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:11.911841 29895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:36101
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:36101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:11.913259 29895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:11.915131 29895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:11.927768 29904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:11.927850 29901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:11.927798 29902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:11.936597 29895 server_base.cc:1034] running on GCE node
I20260809 22:59:11.937716 29895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:11.939357 29895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:11.940647 29895 hybrid_clock.cc:648] HybridClock initialized: now 1786316351940599 us; error 39 us; skew 500 ppm
I20260809 22:59:11.941092 29895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:59:11.943563 29895 webserver.cc:460] Webserver started at http://127.24.31.254:46053/ using document root <none> and password file <none>
I20260809 22:59:11.944291 29895 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:11.944428 29895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:11.944792 29895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:11.947819 29895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "0089e88b6e5c4309a3179f16b247794e"
format_stamp: "Formatted at 2026-08-09 22:59:11 on dist-test-slave-vxj2"
server_key: "663da0a6754d920fc5d598dd7ac1d95d"
server_key_iv: "3ddd05f4f8b1e731dd9bf98c3fa89840"
server_key_version: "encryptionkey@0"
I20260809 22:59:11.948632 29895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "0089e88b6e5c4309a3179f16b247794e"
format_stamp: "Formatted at 2026-08-09 22:59:11 on dist-test-slave-vxj2"
server_key: "663da0a6754d920fc5d598dd7ac1d95d"
server_key_iv: "3ddd05f4f8b1e731dd9bf98c3fa89840"
server_key_version: "encryptionkey@0"
I20260809 22:59:11.953760 29895 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 22:59:11.957077 29910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:11.958479 29895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:59:11.958665 29895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "0089e88b6e5c4309a3179f16b247794e"
format_stamp: "Formatted at 2026-08-09 22:59:11 on dist-test-slave-vxj2"
server_key: "663da0a6754d920fc5d598dd7ac1d95d"
server_key_iv: "3ddd05f4f8b1e731dd9bf98c3fa89840"
server_key_version: "encryptionkey@0"
I20260809 22:59:11.958889 29895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:11.964740 29895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:11.965638 29895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:11.965942 29895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:11.984750 29895 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:36101
I20260809 22:59:11.984724 29970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:36101 every 8 connection(s)
I20260809 22:59:11.986227 29895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:11.988243 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 29895
I20260809 22:59:11.988459 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:11.989136 24703 external_mini_cluster.cc:1428] Setting key 4c178a8c5f67b825efffb2f750ebf377
I20260809 22:59:11.990530 29971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:11.997589 29974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:11.991215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.24.31.254:36101)
0809 22:59:11.992214 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:11.992225 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:11.992273 (+ 48us) server_negotiation.cc:407] Connection header received
0809 22:59:11.993908 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:11.993941 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:11.994347 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:11.994754 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:11.995835 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:11.995841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:11.995855 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:11.995869 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:11.995958 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:11.995980 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:11.996233 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:11.996386 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 22:59:11.996629 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":379,"threads_started":1}
I20260809 22:59:12.003556 29971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e: Bootstrap starting.
I20260809 22:59:12.006552 29971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:12.007547 29971 log.cc:824] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:12.010388 29971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e: No bootstrap required, opened a new log
I20260809 22:59:12.017694 29971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089e88b6e5c4309a3179f16b247794e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 36101 } }
I20260809 22:59:12.018076 29971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:12.018187 29971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0089e88b6e5c4309a3179f16b247794e, State: Initialized, Role: FOLLOWER
I20260809 22:59:12.018731 29971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089e88b6e5c4309a3179f16b247794e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 36101 } }
I20260809 22:59:12.018968 29971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:12.019126 29971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:12.019312 29971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:12.021426 29971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089e88b6e5c4309a3179f16b247794e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 36101 } }
I20260809 22:59:12.021919 29971 leader_election.cc:302] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0089e88b6e5c4309a3179f16b247794e; no voters:
I20260809 22:59:12.022537 29971 leader_election.cc:288] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:12.022688 29977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:12.023594 29977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [term 1 LEADER]: Becoming Leader. State: Replica: 0089e88b6e5c4309a3179f16b247794e, State: Running, Role: LEADER
I20260809 22:59:12.024326 29977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089e88b6e5c4309a3179f16b247794e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 36101 } }
I20260809 22:59:12.024824 29971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:12.027179 29980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0089e88b6e5c4309a3179f16b247794e. Latest consensus state: current_term: 1 leader_uuid: "0089e88b6e5c4309a3179f16b247794e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089e88b6e5c4309a3179f16b247794e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 36101 } } }
I20260809 22:59:12.027549 29980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:12.028231 29979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0089e88b6e5c4309a3179f16b247794e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089e88b6e5c4309a3179f16b247794e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 36101 } } }
I20260809 22:59:12.028631 29979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:12.031379 29984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:12.036856 29984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:12.045972 29984 catalog_manager.cc:1347] Generated new cluster ID: 167769dc6a78432e9b08fe73ca13e2ea
I20260809 22:59:12.046096 29984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:12.112574 29984 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:12.113041 29984 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:12.121595 29984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e: Generated new TSK 0
I20260809 22:59:12.122406 29984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:59:12.129204 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:36101
--builtin_ntp_servers=127.24.31.212:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 22:59:12.485437 29999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:12.485773 29999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:12.485961 29999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:12.493517 29999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:12.493654 29999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:12.493858 29999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:12.503558 29999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:12.505007 29999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:12.506906 29999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:12.520565 30005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:12.520658 30006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:12.520609 30008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:12.522496 29999 server_base.cc:1034] running on GCE node
I20260809 22:59:12.523154 29999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:12.524132 29999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:12.525359 29999 hybrid_clock.cc:648] HybridClock initialized: now 1786316352525319 us; error 52 us; skew 500 ppm
I20260809 22:59:12.525687 29999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:59:12.527920 29999 webserver.cc:460] Webserver started at http://127.24.31.193:33475/ using document root <none> and password file <none>
I20260809 22:59:12.528606 29999 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:12.528749 29999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:12.529093 29999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:12.532193 29999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "91b3f3c9b63d40948d7353f13ddc5029"
format_stamp: "Formatted at 2026-08-09 22:59:12 on dist-test-slave-vxj2"
server_key: "7dad876ea20c262eb9d00ed9448729f8"
server_key_iv: "b92d5dcdb4791d363714da594a9939eb"
server_key_version: "encryptionkey@0"
I20260809 22:59:12.532994 29999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "91b3f3c9b63d40948d7353f13ddc5029"
format_stamp: "Formatted at 2026-08-09 22:59:12 on dist-test-slave-vxj2"
server_key: "7dad876ea20c262eb9d00ed9448729f8"
server_key_iv: "b92d5dcdb4791d363714da594a9939eb"
server_key_version: "encryptionkey@0"
I20260809 22:59:12.538491 29999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 22:59:12.542049 30014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:12.543321 29999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:59:12.543516 29999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "91b3f3c9b63d40948d7353f13ddc5029"
format_stamp: "Formatted at 2026-08-09 22:59:12 on dist-test-slave-vxj2"
server_key: "7dad876ea20c262eb9d00ed9448729f8"
server_key_iv: "b92d5dcdb4791d363714da594a9939eb"
server_key_version: "encryptionkey@0"
I20260809 22:59:12.543769 29999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:12.549494 29999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:12.550294 29999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:12.550602 29999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:12.551524 29999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:12.552902 29999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:12.553035 29999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:12.553195 29999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:12.553308 29999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 22:59:12.587395 29999 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:37517
I20260809 22:59:12.587458 30139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:37517 every 8 connection(s)
I20260809 22:59:12.588769 29999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:12.595103 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 29999
I20260809 22:59:12.595326 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:59:12.596172 24703 external_mini_cluster.cc:1428] Setting key 5787ad4488260c0493fa24f36ead03d2
I20260809 22:59:12.597777 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:12.591471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:41721 (local address 127.24.31.254:36101)
0809 22:59:12.591909 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:12.591913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:12.593137 (+ 1224us) server_negotiation.cc:407] Connection header received
0809 22:59:12.594947 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:12.594951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:12.595015 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:12.595116 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:12.596678 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:12.596682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:12.596685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:12.596703 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:12.596746 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:12.596758 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:12.596890 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:12.597373 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 22:59:12.597439 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":228,"threads_started":1}
I20260809 22:59:12.598770 30143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:12.592099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:36101 (local address 127.24.31.193:41721)
0809 22:59:12.592978 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:12.593017 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:12.594634 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:12.595412 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:12.595425 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:12.595560 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:12.596311 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:12.596316 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:12.596371 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:12.596494 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:12.597107 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:12.597119 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:12.597127 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:59:12.597423 (+ 296us) client_negotiation.cc:240] Negotiation successful
0809 22:59:12.597622 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":276,"threads_started":1}
I20260809 22:59:12.599449 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:36101
--builtin_ntp_servers=127.24.31.212:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 22:59:12.600291 30140 heartbeater.cc:347] Connected to a master server at 127.24.31.254:36101
I20260809 22:59:12.600630 30140 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:12.601342 30140 heartbeater.cc:511] Master 127.24.31.254:36101 requested a full tablet report, sending...
I20260809 22:59:12.603410 29930 ts_manager.cc:194] Registered new tserver with Master: 91b3f3c9b63d40948d7353f13ddc5029 (127.24.31.193:37517)
W20260809 22:59:12.926024 30144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:12.926326 30144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:12.926537 30144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:12.933907 30144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:12.934033 30144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:12.934199 30144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:12.943218 30144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:12.944473 30144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:12.946281 30144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:12.959370 30151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:12.959345 30152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:12.959501 30154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:12.960523 30144 server_base.cc:1034] running on GCE node
I20260809 22:59:12.961117 30144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:12.962121 30144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:12.963316 30144 hybrid_clock.cc:648] HybridClock initialized: now 1786316352963267 us; error 57 us; skew 500 ppm
I20260809 22:59:12.963685 30144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:59:12.965920 30144 webserver.cc:460] Webserver started at http://127.24.31.194:40919/ using document root <none> and password file <none>
I20260809 22:59:12.966574 30144 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:12.966662 30144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:12.966924 30144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:12.969647 30144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "789a2dcf6d984b8e8d31e2d93244efb8"
format_stamp: "Formatted at 2026-08-09 22:59:12 on dist-test-slave-vxj2"
server_key: "b0361045a84a7d4962322279a92ed4f3"
server_key_iv: "ed9c3ac2f653e5ce967355299a670508"
server_key_version: "encryptionkey@0"
I20260809 22:59:12.970546 30144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "789a2dcf6d984b8e8d31e2d93244efb8"
format_stamp: "Formatted at 2026-08-09 22:59:12 on dist-test-slave-vxj2"
server_key: "b0361045a84a7d4962322279a92ed4f3"
server_key_iv: "ed9c3ac2f653e5ce967355299a670508"
server_key_version: "encryptionkey@0"
I20260809 22:59:12.975641 30144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 22:59:12.978886 30161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:12.980105 30144 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 22:59:12.980257 30144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "789a2dcf6d984b8e8d31e2d93244efb8"
format_stamp: "Formatted at 2026-08-09 22:59:12 on dist-test-slave-vxj2"
server_key: "b0361045a84a7d4962322279a92ed4f3"
server_key_iv: "ed9c3ac2f653e5ce967355299a670508"
server_key_version: "encryptionkey@0"
I20260809 22:59:12.980476 30144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:12.986202 30144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:12.986969 30144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:12.987238 30144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:12.988160 30144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:12.989650 30144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:12.989763 30144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:12.989886 30144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:12.989981 30144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:13.024106 30144 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:37361
I20260809 22:59:13.024156 30282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:37361 every 8 connection(s)
I20260809 22:59:13.025626 30144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:13.031395 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 30144
I20260809 22:59:13.031574 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:59:13.032229 24703 external_mini_cluster.cc:1428] Setting key 9a1c3a6f82605763481808538304fed9
I20260809 22:59:13.034060 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.028182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:38913 (local address 127.24.31.254:36101)
0809 22:59:13.028377 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.028382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.029705 (+ 1323us) server_negotiation.cc:407] Connection header received
0809 22:59:13.031577 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.031588 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.031703 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.031798 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.033242 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.033248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.033253 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.033272 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.033341 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.033370 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.033499 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.033727 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.033784 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:59:13.035202 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.028646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:36101 (local address 127.24.31.194:38913)
0809 22:59:13.029561 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:13.029596 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:13.031249 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:13.032054 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:13.032066 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:13.032188 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:13.032946 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:13.032949 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:13.032985 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:13.033063 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:13.033496 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:13.033507 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:13.033516 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:59:13.033854 (+ 338us) client_negotiation.cc:240] Negotiation successful
0809 22:59:13.034084 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":268,"threads_started":1}
I20260809 22:59:13.035642 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:36101
--builtin_ntp_servers=127.24.31.212:36329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 22:59:13.036720 30283 heartbeater.cc:347] Connected to a master server at 127.24.31.254:36101
I20260809 22:59:13.037071 30283 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:13.037614 30283 heartbeater.cc:511] Master 127.24.31.254:36101 requested a full tablet report, sending...
I20260809 22:59:13.038950 29930 ts_manager.cc:194] Registered new tserver with Master: 789a2dcf6d984b8e8d31e2d93244efb8 (127.24.31.194:37361)
W20260809 22:59:13.361730 30286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:13.361997 30286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:13.362166 30286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:13.369249 30286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:13.369359 30286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:13.369540 30286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:13.378835 30286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:36101
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:13.380179 30286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:13.382846 30286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:13.399569 30291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:13.399544 30292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:13.399806 30294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:13.400336 30286 server_base.cc:1034] running on GCE node
I20260809 22:59:13.400967 30286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:13.401950 30286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:13.403177 30286 hybrid_clock.cc:648] HybridClock initialized: now 1786316353403135 us; error 36 us; skew 500 ppm
I20260809 22:59:13.403496 30286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:59:13.405781 30286 webserver.cc:460] Webserver started at http://127.24.31.195:39753/ using document root <none> and password file <none>
I20260809 22:59:13.406445 30286 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:13.406595 30286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:13.406925 30286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:13.409865 30286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c"
format_stamp: "Formatted at 2026-08-09 22:59:13 on dist-test-slave-vxj2"
server_key: "66d66e15a295dbb617c708d524bad427"
server_key_iv: "f13026b0162221bac0f834c7f9d8abb3"
server_key_version: "encryptionkey@0"
I20260809 22:59:13.410647 30286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c"
format_stamp: "Formatted at 2026-08-09 22:59:13 on dist-test-slave-vxj2"
server_key: "66d66e15a295dbb617c708d524bad427"
server_key_iv: "f13026b0162221bac0f834c7f9d8abb3"
server_key_version: "encryptionkey@0"
I20260809 22:59:13.415577 30286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:59:13.418844 30301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:13.420316 30286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:59:13.420599 30286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c"
format_stamp: "Formatted at 2026-08-09 22:59:13 on dist-test-slave-vxj2"
server_key: "66d66e15a295dbb617c708d524bad427"
server_key_iv: "f13026b0162221bac0f834c7f9d8abb3"
server_key_version: "encryptionkey@0"
I20260809 22:59:13.420897 30286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:13.426614 30286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:13.427387 30286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:13.427660 30286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:13.428526 30286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:13.430011 30286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:13.430133 30286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:13.430274 30286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:13.430365 30286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:13.465729 30286 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:38299
I20260809 22:59:13.465787 30424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:38299 every 8 connection(s)
I20260809 22:59:13.467073 30286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:13.470198 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 30286
I20260809 22:59:13.470369 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:59:13.471024 24703 external_mini_cluster.cc:1428] Setting key 4cfc443f88bff19c3ded22ff0e90fe0d
I20260809 22:59:13.475868 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.469736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:57961 (local address 127.24.31.254:36101)
0809 22:59:13.469918 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.469922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.471728 (+ 1806us) server_negotiation.cc:407] Connection header received
0809 22:59:13.473688 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.473693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.473758 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.473869 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.475265 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.475270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.475273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.475291 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.475325 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.475337 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.475427 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.475601 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.475651 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:59:13.476754 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.470407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:36101 (local address 127.24.31.195:57961)
0809 22:59:13.471516 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:13.471583 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:13.473428 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:13.474273 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:13.474281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:13.474374 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:13.474978 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:13.474981 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:13.475018 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:13.475105 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:13.475424 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:13.475432 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:13.475437 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:59:13.475645 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 22:59:13.475841 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":365,"threads_started":1}
I20260809 22:59:13.478041 30425 heartbeater.cc:347] Connected to a master server at 127.24.31.254:36101
I20260809 22:59:13.478338 30425 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:13.478886 30425 heartbeater.cc:511] Master 127.24.31.254:36101 requested a full tablet report, sending...
I20260809 22:59:13.480252 29930 ts_manager.cc:194] Registered new tserver with Master: 4e55a2c3fb204eb9bc7ee524c59fa16c (127.24.31.195:38299)
I20260809 22:59:13.486882 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 22:59:13.494588 30142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.491950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58534 (local address 127.24.31.254:36101)
0809 22:59:13.492128 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.492133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.492735 (+ 602us) server_negotiation.cc:407] Connection header received
0809 22:59:13.492859 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.492863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.492918 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.493011 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.494012 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.494018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.494021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.494040 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.494094 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.494109 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.494228 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.494301 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.494355 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:59:13.498843 29930 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:59:13.505851 29930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:59:13.535391 30440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.517788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:38299 (local address 127.0.0.1:44814)
0809 22:59:13.531375 (+ 13587us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:13.531426 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:13.531592 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:13.532357 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:13.532365 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:13.532455 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:13.533425 (+ 970us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:13.533429 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:13.533464 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:13.533580 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:13.534538 (+ 958us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:13.534551 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:13.534559 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:59:13.534715 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 22:59:13.534819 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13338,"thread_start_us":15598,"threads_started":1}
I20260809 22:59:13.535843 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.518753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44814 (local address 127.24.31.195:38299)
0809 22:59:13.529860 (+ 11107us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.529873 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.531750 (+ 1877us) server_negotiation.cc:407] Connection header received
0809 22:59:13.531854 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.531865 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.532133 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.533012 (+ 879us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.534255 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.534264 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.534278 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.534298 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.534367 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.534390 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.535216 (+ 826us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.535285 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.535392 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10887,"thread_start_us":10694,"threads_started":1}
I20260809 22:59:13.536495 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.519403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.24.31.193:37517)
0809 22:59:13.530324 (+ 10921us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.530335 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.532341 (+ 2006us) server_negotiation.cc:407] Connection header received
0809 22:59:13.532416 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.532423 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.532644 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.532829 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.534400 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.534408 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.534422 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.534475 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.534552 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.534574 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.534716 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.536039 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.536146 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10737,"thread_start_us":10601,"threads_started":1}
I20260809 22:59:13.537179 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.518199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37517 (local address 127.0.0.1:49346)
0809 22:59:13.532078 (+ 13879us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:13.532097 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:13.532207 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:13.533063 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:13.533067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:13.533088 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:13.534191 (+ 1103us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:13.534193 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:13.534206 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:13.534240 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:13.534775 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:13.534780 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:13.534785 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:59:13.534946 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 22:59:13.535006 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13742,"thread_start_us":12779,"threads_started":1}
I20260809 22:59:13.537768 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.519977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36952 (local address 127.24.31.194:37361)
0809 22:59:13.530787 (+ 10810us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.530795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.533360 (+ 2565us) server_negotiation.cc:407] Connection header received
0809 22:59:13.534129 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.534147 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.534365 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.534518 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.536848 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.536859 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.536873 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.536901 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.536971 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.536995 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.537141 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.537364 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.537464 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10642,"thread_start_us":10505,"threads_started":1}
I20260809 22:59:13.538583 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.518517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:37361 (local address 127.0.0.1:36952)
0809 22:59:13.532869 (+ 14352us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:13.532908 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:13.533661 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:13.535518 (+ 1857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:13.535523 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:13.535547 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:13.536584 (+ 1037us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:13.536587 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:13.536603 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:13.536644 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:13.537148 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:13.537153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:13.537158 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 22:59:13.538098 (+ 940us) client_negotiation.cc:240] Negotiation successful
0809 22:59:13.538159 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14246,"thread_start_us":13296,"threads_started":1}
I20260809 22:59:13.540256 30355 tablet_service.cc:1467] Processing CreateTablet for tablet 94e26e4ab2984570b92fb6fe921b7049 (DEFAULT_TABLE table=test-table [id=53f4192568764a43bfd958a7a4cc8312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:13.540664 30069 tablet_service.cc:1467] Processing CreateTablet for tablet 94e26e4ab2984570b92fb6fe921b7049 (DEFAULT_TABLE table=test-table [id=53f4192568764a43bfd958a7a4cc8312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:13.541858 30355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94e26e4ab2984570b92fb6fe921b7049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:13.542140 30069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94e26e4ab2984570b92fb6fe921b7049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:13.542392 30213 tablet_service.cc:1467] Processing CreateTablet for tablet 94e26e4ab2984570b92fb6fe921b7049 (DEFAULT_TABLE table=test-table [id=53f4192568764a43bfd958a7a4cc8312]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:13.543789 30213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94e26e4ab2984570b92fb6fe921b7049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:13.554114 30446 tablet_bootstrap.cc:492] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: Bootstrap starting.
I20260809 22:59:13.557199 30447 tablet_bootstrap.cc:492] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: Bootstrap starting.
I20260809 22:59:13.558832 30446 tablet_bootstrap.cc:654] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:13.560163 30446 log.cc:824] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:13.560961 30447 tablet_bootstrap.cc:654] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:13.561671 30448 tablet_bootstrap.cc:492] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: Bootstrap starting.
I20260809 22:59:13.562318 30447 log.cc:824] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:13.563825 30446 tablet_bootstrap.cc:492] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: No bootstrap required, opened a new log
I20260809 22:59:13.564129 30446 ts_tablet_manager.cc:1397] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 22:59:13.565568 30448 tablet_bootstrap.cc:654] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:13.566740 30448 log.cc:824] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:13.567790 30447 tablet_bootstrap.cc:492] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: No bootstrap required, opened a new log
I20260809 22:59:13.568104 30447 ts_tablet_manager.cc:1397] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260809 22:59:13.569723 30448 tablet_bootstrap.cc:492] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: No bootstrap required, opened a new log
I20260809 22:59:13.569981 30448 ts_tablet_manager.cc:1397] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 22:59:13.572255 30446 raft_consensus.cc:348] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.572801 30446 raft_consensus.cc:374] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:13.572973 30446 raft_consensus.cc:729] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e55a2c3fb204eb9bc7ee524c59fa16c, State: Initialized, Role: FOLLOWER
I20260809 22:59:13.573622 30446 consensus_queue.cc:260] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.574764 30425 heartbeater.cc:503] Master 127.24.31.254:36101 was elected leader, sending a full tablet report...
I20260809 22:59:13.576475 30446 ts_tablet_manager.cc:1428] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260809 22:59:13.579007 30447 raft_consensus.cc:348] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.579471 30447 raft_consensus.cc:374] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:13.579593 30447 raft_consensus.cc:729] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b3f3c9b63d40948d7353f13ddc5029, State: Initialized, Role: FOLLOWER
I20260809 22:59:13.579909 30448 raft_consensus.cc:348] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.580358 30448 raft_consensus.cc:374] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:13.580479 30448 raft_consensus.cc:729] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 789a2dcf6d984b8e8d31e2d93244efb8, State: Initialized, Role: FOLLOWER
I20260809 22:59:13.580379 30447 consensus_queue.cc:260] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.581113 30448 consensus_queue.cc:260] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.581640 30140 heartbeater.cc:503] Master 127.24.31.254:36101 was elected leader, sending a full tablet report...
I20260809 22:59:13.582304 30283 heartbeater.cc:503] Master 127.24.31.254:36101 was elected leader, sending a full tablet report...
I20260809 22:59:13.582389 30447 ts_tablet_manager.cc:1428] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260809 22:59:13.583825 30448 ts_tablet_manager.cc:1428] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260809 22:59:13.591143 30141 tablet.cc:2369] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:59:13.718915 30426 tablet.cc:2369] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:59:13.777684 30284 tablet.cc:2369] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:59:13.814177 30458 raft_consensus.cc:482] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:13.814503 30458 raft_consensus.cc:504] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.815218 30457 raft_consensus.cc:482] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:13.815552 30457 raft_consensus.cc:504] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.816432 30458 leader_election.cc:288] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e55a2c3fb204eb9bc7ee524c59fa16c (127.24.31.195:38299), 91b3f3c9b63d40948d7353f13ddc5029 (127.24.31.193:37517)
I20260809 22:59:13.817338 30457 leader_election.cc:288] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e55a2c3fb204eb9bc7ee524c59fa16c (127.24.31.195:38299), 789a2dcf6d984b8e8d31e2d93244efb8 (127.24.31.194:37361)
I20260809 22:59:13.822311 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.817914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:35199 (local address 127.24.31.195:38299)
0809 22:59:13.818138 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.818146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.819736 (+ 1590us) server_negotiation.cc:407] Connection header received
0809 22:59:13.819994 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.819999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.820061 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.820143 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.821523 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.821532 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.821540 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.821562 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.821598 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.821614 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.821746 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.821958 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.822023 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 22:59:13.822325 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.817306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:40379 (local address 127.24.31.193:37517)
0809 22:59:13.817593 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.817603 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.819378 (+ 1775us) server_negotiation.cc:407] Connection header received
0809 22:59:13.819656 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.819664 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.819734 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.819811 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.821587 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.821592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.821596 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.821614 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.821649 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.821667 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.821779 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.821929 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.821992 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 22:59:13.822986 30463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.817623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:38299 (local address 127.24.31.193:52853)
0809 22:59:13.818868 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:13.818894 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:13.819039 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:13.820425 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:13.820432 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:13.820459 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:13.821905 (+ 1446us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:13.821910 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:13.821928 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:13.821983 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:13.822324 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:13.822331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:13.822342 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 22:59:13.822489 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 22:59:13.822551 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"thread_start_us":3180,"threads_started":1}
I20260809 22:59:13.823091 30459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.816838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37517 (local address 127.24.31.194:40379)
0809 22:59:13.819249 (+ 2411us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:13.819280 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:13.819490 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:13.820082 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:13.820089 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:13.820122 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:13.821280 (+ 1158us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:13.821283 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:13.821294 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:13.821321 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:13.821761 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:13.821777 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:13.821781 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 22:59:13.822587 (+ 806us) client_negotiation.cc:240] Negotiation successful
0809 22:59:13.822646 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2251,"thread_start_us":2050,"threads_started":1}
I20260809 22:59:13.824198 30461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.816863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:38299 (local address 127.24.31.194:35199)
0809 22:59:13.819627 (+ 2764us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:13.819649 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:13.819831 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:13.820360 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:13.820366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:13.820397 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:13.821226 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:13.821231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:13.821256 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:13.821305 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:13.821804 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:13.821811 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:13.821818 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 22:59:13.822969 (+ 1151us) client_negotiation.cc:240] Negotiation successful
0809 22:59:13.823032 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2635,"thread_start_us":2472,"threads_started":1}
I20260809 22:59:13.824219 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.818257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:51911 (local address 127.24.31.194:37361)
0809 22:59:13.818480 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.818488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.821592 (+ 3104us) server_negotiation.cc:407] Connection header received
0809 22:59:13.821801 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.821806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.821866 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.821928 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.823470 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.823476 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.823481 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.823499 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.823532 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.823546 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.823630 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.823876 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.823930 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 22:59:13.824183 30089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e26e4ab2984570b92fb6fe921b7049" candidate_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b3f3c9b63d40948d7353f13ddc5029" is_pre_election: true
I20260809 22:59:13.824658 30462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.817823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:37361 (local address 127.24.31.193:51911)
0809 22:59:13.821143 (+ 3320us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:13.821500 (+ 357us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:13.821671 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:13.822098 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:13.822102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:13.822126 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:13.823249 (+ 1123us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 22:59:13.823253 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:13.823269 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 22:59:13.823311 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:13.823727 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:13.823733 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:13.823739 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 22:59:13.823885 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 22:59:13.823997 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3152,"thread_start_us":674,"threads_started":1}
I20260809 22:59:13.824831 30089 raft_consensus.cc:2454] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 789a2dcf6d984b8e8d31e2d93244efb8 in term 0.
I20260809 22:59:13.825026 30464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.818086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:52853 (local address 127.24.31.195:38299)
0809 22:59:13.820133 (+ 2047us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.820138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.820158 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:59:13.820205 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.820209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.820255 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.821080 (+ 825us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.822124 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.822129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.822134 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.822149 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.822188 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.822201 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.823233 (+ 1032us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.823284 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.823337 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1917,"thread_start_us":1700,"threads_started":1}
I20260809 22:59:13.825507 30376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e26e4ab2984570b92fb6fe921b7049" candidate_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" is_pre_election: true
I20260809 22:59:13.825654 30375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e26e4ab2984570b92fb6fe921b7049" candidate_uuid: "91b3f3c9b63d40948d7353f13ddc5029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" is_pre_election: true
I20260809 22:59:13.825866 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e26e4ab2984570b92fb6fe921b7049" candidate_uuid: "91b3f3c9b63d40948d7353f13ddc5029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" is_pre_election: true
I20260809 22:59:13.826174 30376 raft_consensus.cc:2454] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 789a2dcf6d984b8e8d31e2d93244efb8 in term 0.
I20260809 22:59:13.826220 30163 leader_election.cc:302] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 789a2dcf6d984b8e8d31e2d93244efb8, 91b3f3c9b63d40948d7353f13ddc5029; no voters:
I20260809 22:59:13.826318 30235 raft_consensus.cc:2454] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b3f3c9b63d40948d7353f13ddc5029 in term 0.
I20260809 22:59:13.826740 30458 raft_consensus.cc:2781] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:13.826900 30458 raft_consensus.cc:482] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:13.827028 30458 raft_consensus.cc:3037] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:13.827015 30019 leader_election.cc:302] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 789a2dcf6d984b8e8d31e2d93244efb8, 91b3f3c9b63d40948d7353f13ddc5029; no voters:
I20260809 22:59:13.827395 30457 raft_consensus.cc:2781] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:13.827548 30457 raft_consensus.cc:482] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:13.827687 30457 raft_consensus.cc:3037] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:13.829914 30458 raft_consensus.cc:504] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.830109 30457 raft_consensus.cc:504] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.830560 30458 leader_election.cc:288] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [CANDIDATE]: Term 1 election: Requested vote from peers 4e55a2c3fb204eb9bc7ee524c59fa16c (127.24.31.195:38299), 91b3f3c9b63d40948d7353f13ddc5029 (127.24.31.193:37517)
I20260809 22:59:13.830914 30457 leader_election.cc:288] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [CANDIDATE]: Term 1 election: Requested vote from peers 4e55a2c3fb204eb9bc7ee524c59fa16c (127.24.31.195:38299), 789a2dcf6d984b8e8d31e2d93244efb8 (127.24.31.194:37361)
I20260809 22:59:13.830988 30375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e26e4ab2984570b92fb6fe921b7049" candidate_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c"
I20260809 22:59:13.831089 30089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e26e4ab2984570b92fb6fe921b7049" candidate_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b3f3c9b63d40948d7353f13ddc5029"
I20260809 22:59:13.831228 30375 raft_consensus.cc:3037] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:13.831382 30089 raft_consensus.cc:2379] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 789a2dcf6d984b8e8d31e2d93244efb8 in current term 1: Already voted for candidate 91b3f3c9b63d40948d7353f13ddc5029 in this term.
I20260809 22:59:13.831297 30376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e26e4ab2984570b92fb6fe921b7049" candidate_uuid: "91b3f3c9b63d40948d7353f13ddc5029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c"
I20260809 22:59:13.832023 30235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94e26e4ab2984570b92fb6fe921b7049" candidate_uuid: "91b3f3c9b63d40948d7353f13ddc5029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "789a2dcf6d984b8e8d31e2d93244efb8"
I20260809 22:59:13.832289 30235 raft_consensus.cc:2379] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91b3f3c9b63d40948d7353f13ddc5029 in current term 1: Already voted for candidate 789a2dcf6d984b8e8d31e2d93244efb8 in this term.
I20260809 22:59:13.832877 30019 leader_election.cc:302] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91b3f3c9b63d40948d7353f13ddc5029; no voters: 4e55a2c3fb204eb9bc7ee524c59fa16c, 789a2dcf6d984b8e8d31e2d93244efb8
I20260809 22:59:13.833173 30457 raft_consensus.cc:2726] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 22:59:13.834080 30375 raft_consensus.cc:2454] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 789a2dcf6d984b8e8d31e2d93244efb8 in term 1.
I20260809 22:59:13.834486 30164 leader_election.cc:302] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e55a2c3fb204eb9bc7ee524c59fa16c, 789a2dcf6d984b8e8d31e2d93244efb8; no voters: 91b3f3c9b63d40948d7353f13ddc5029
I20260809 22:59:13.834805 30458 raft_consensus.cc:2781] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:13.835355 30458 raft_consensus.cc:686] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 1 LEADER]: Becoming Leader. State: Replica: 789a2dcf6d984b8e8d31e2d93244efb8, State: Running, Role: LEADER
I20260809 22:59:13.835861 30458 consensus_queue.cc:237] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } }
I20260809 22:59:13.840399 29930 catalog_manager.cc:5495] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 789a2dcf6d984b8e8d31e2d93244efb8 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "789a2dcf6d984b8e8d31e2d93244efb8" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37361 } health_report { overall_health: HEALTHY } } }
I20260809 22:59:13.909588 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.907048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36964 (local address 127.24.31.194:37361)
0809 22:59:13.907297 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.907304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.907415 (+ 111us) server_negotiation.cc:407] Connection header received
0809 22:59:13.907686 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.907690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.907802 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.907922 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.908872 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.908878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.908889 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.908906 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.908940 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.908952 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.909088 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.909259 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.909322 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 22:59:13.917799 30375 raft_consensus.cc:1261] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [term 1 FOLLOWER]: Refusing update from remote peer 789a2dcf6d984b8e8d31e2d93244efb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:13.917816 30089 raft_consensus.cc:1261] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 1 FOLLOWER]: Refusing update from remote peer 789a2dcf6d984b8e8d31e2d93244efb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:13.918723 30465 consensus_queue.cc:1035] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b3f3c9b63d40948d7353f13ddc5029" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:13.919143 30458 consensus_queue.cc:1035] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e55a2c3fb204eb9bc7ee524c59fa16c" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:13.945659 29928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:13.945984 29928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58534:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:13.949937 29928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0089e88b6e5c4309a3179f16b247794e: Sending DeleteTablet for 3 replicas of tablet 94e26e4ab2984570b92fb6fe921b7049
I20260809 22:59:13.951086 30069 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e26e4ab2984570b92fb6fe921b7049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:13 UTC) from {username='slave'} at 127.0.0.1:49346
I20260809 22:59:13.951529 30213 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e26e4ab2984570b92fb6fe921b7049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:13 UTC) from {username='slave'} at 127.0.0.1:36952
I20260809 22:59:13.951865 30355 tablet_service.cc:1514] Processing DeleteTablet for tablet 94e26e4ab2984570b92fb6fe921b7049 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:13 UTC) from {username='slave'} at 127.0.0.1:44814
I20260809 22:59:13.952050 30479 tablet_replica.cc:330] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: stopping tablet replica
I20260809 22:59:13.952491 30479 raft_consensus.cc:2229] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:13.953073 30479 raft_consensus.cc:2258] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:13.953759 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 29999
I20260809 22:59:13.955471 30479 ts_tablet_manager.cc:1905] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:13.964915 30479 ts_tablet_manager.cc:1918] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:13.964975 30480 tablet_replica.cc:330] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: stopping tablet replica
I20260809 22:59:13.965173 30479 log.cc:1196] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/94e26e4ab2984570b92fb6fe921b7049
I20260809 22:59:13.965695 30480 raft_consensus.cc:2229] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:59:13.965790 30479 ts_tablet_manager.cc:1939] T 94e26e4ab2984570b92fb6fe921b7049 P 91b3f3c9b63d40948d7353f13ddc5029: Deleting consensus metadata
I20260809 22:59:13.966463 30480 raft_consensus.cc:2258] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:13.967839 30481 tablet_replica.cc:330] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: stopping tablet replica
I20260809 22:59:13.967782 29913 catalog_manager.cc:4841] TS 91b3f3c9b63d40948d7353f13ddc5029 (127.24.31.193:37517): tablet 94e26e4ab2984570b92fb6fe921b7049 (table test-table [id=53f4192568764a43bfd958a7a4cc8312]) successfully deleted
I20260809 22:59:13.969368 30480 ts_tablet_manager.cc:1905] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:13.972625 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:13.955483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49356 (local address 127.24.31.193:37517)
0809 22:59:13.955669 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:13.955675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:13.969919 (+ 14244us) server_negotiation.cc:407] Connection header received
0809 22:59:13.970397 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:13.970403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:13.970522 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:13.970635 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:13.971953 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:13.971972 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:13.971977 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:13.971996 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:13.972033 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:13.972047 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:13.972171 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:13.972295 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 22:59:13.972353 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:59:13.973453 30134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:13.975513 30481 raft_consensus.cc:2229] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:13.976161 30481 raft_consensus.cc:2258] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:13.978551 30481 ts_tablet_manager.cc:1905] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:13.979159 30480 ts_tablet_manager.cc:1918] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:13.979405 30480 log.cc:1196] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/94e26e4ab2984570b92fb6fe921b7049
I20260809 22:59:13.980000 30480 ts_tablet_manager.cc:1939] T 94e26e4ab2984570b92fb6fe921b7049 P 789a2dcf6d984b8e8d31e2d93244efb8: Deleting consensus metadata
I20260809 22:59:13.981796 29915 catalog_manager.cc:4841] TS 789a2dcf6d984b8e8d31e2d93244efb8 (127.24.31.194:37361): tablet 94e26e4ab2984570b92fb6fe921b7049 (table test-table [id=53f4192568764a43bfd958a7a4cc8312]) successfully deleted
I20260809 22:59:13.985038 30481 ts_tablet_manager.cc:1918] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:13.985291 30481 log.cc:1196] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/94e26e4ab2984570b92fb6fe921b7049
I20260809 22:59:13.985818 30481 ts_tablet_manager.cc:1939] T 94e26e4ab2984570b92fb6fe921b7049 P 4e55a2c3fb204eb9bc7ee524c59fa16c: Deleting consensus metadata
I20260809 22:59:13.987521 29914 catalog_manager.cc:4841] TS 4e55a2c3fb204eb9bc7ee524c59fa16c (127.24.31.195:38299): tablet 94e26e4ab2984570b92fb6fe921b7049 (table test-table [id=53f4192568764a43bfd958a7a4cc8312]) successfully deleted
I20260809 22:59:14.116663 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 29999
I20260809 22:59:14.136246 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 30144
I20260809 22:59:14.139362 30445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:14.137044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.24.31.194:37361)
0809 22:59:14.137240 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:14.137247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:14.137334 (+ 87us) server_negotiation.cc:407] Connection header received
0809 22:59:14.137550 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:14.137556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:14.137614 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:14.137728 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:14.138648 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:14.138653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:14.138658 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:14.138677 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:14.138716 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:14.138728 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:14.138867 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:14.138999 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 22:59:14.139076 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:59:14.140049 30278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:14.257746 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 30144
I20260809 22:59:14.280273 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 30286
I20260809 22:59:14.283318 30464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:14.281054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.24.31.195:38299)
0809 22:59:14.281297 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:14.281303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:14.281324 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:59:14.281534 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:14.281541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:14.281647 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:14.281759 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:14.282693 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:14.282699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:14.282704 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:14.282723 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:14.282764 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:14.282775 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:14.282910 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:14.282999 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 22:59:14.283066 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:59:14.284027 30420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:14.374039 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 30286
I20260809 22:59:14.395228 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 29895
I20260809 22:59:14.396540 29966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:14.507424 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 29895
2026-08-09T22:59:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3030 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 22:59:14.552099 24703 mini_oidc.cc:152] Starting JWKS server
I20260809 22:59:14.555091 24703 webserver.cc:460] Webserver started at https://127.0.0.1:43143/ using document root <none> and password file <none>
I20260809 22:59:14.555516 24703 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:59:14.556231 24703 webserver.cc:460] Webserver started at http://127.0.0.1:46015/ using document root <none> and password file <none>
2026-08-09T22:59:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:14Z Disabled control of system clock
I20260809 22:59:14.587117 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39651
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:37165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39651
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43143/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/testchainca.cert with env {}
W20260809 22:59:14.908265 30494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:14.908617 30494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:14.908774 30494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:14.915974 30494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:14.916118 30494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:14.916206 30494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:14.916312 30494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:14.916385 30494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:59:14.916481 30494 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:59:14.916553 30494 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43143/jwks.json
I20260809 22:59:14.926501 30494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:37165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39651
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43143/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:14.927802 30494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:14.929131 30494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:14.941215 30500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:14.941354 30499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:14.941378 30502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:14.942430 30494 server_base.cc:1034] running on GCE node
I20260809 22:59:14.943252 30494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:14.944738 30494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:14.945968 30494 hybrid_clock.cc:648] HybridClock initialized: now 1786316354945928 us; error 35 us; skew 500 ppm
I20260809 22:59:14.946328 30494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:59:14.953774 30494 webserver.cc:460] Webserver started at http://127.24.31.254:42227/ using document root <none> and password file <none>
I20260809 22:59:14.954516 30494 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:14.954668 30494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:14.955001 30494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:14.958081 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "53864b864d1846a1af7c16dc082fb91b"
format_stamp: "Formatted at 2026-08-09 22:59:14 on dist-test-slave-vxj2"
server_key: "4fe77954829bf3128114e451bab35533"
server_key_iv: "ae4d70f9dad8fd3584dfb9b086bd1271"
server_key_version: "encryptionkey@0"
I20260809 22:59:14.958900 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "53864b864d1846a1af7c16dc082fb91b"
format_stamp: "Formatted at 2026-08-09 22:59:14 on dist-test-slave-vxj2"
server_key: "4fe77954829bf3128114e451bab35533"
server_key_iv: "ae4d70f9dad8fd3584dfb9b086bd1271"
server_key_version: "encryptionkey@0"
I20260809 22:59:14.963999 30494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:59:14.967466 30510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:14.968752 30494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:59:14.968937 30494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "53864b864d1846a1af7c16dc082fb91b"
format_stamp: "Formatted at 2026-08-09 22:59:14 on dist-test-slave-vxj2"
server_key: "4fe77954829bf3128114e451bab35533"
server_key_iv: "ae4d70f9dad8fd3584dfb9b086bd1271"
server_key_version: "encryptionkey@0"
I20260809 22:59:14.969195 30494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:14.990551 30494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:14.991393 30494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:14.991730 30494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:15.010742 30494 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:39651
I20260809 22:59:15.010807 30566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:39651 every 8 connection(s)
I20260809 22:59:15.012136 30494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:15.016014 30567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:15.016098 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 30494
I20260809 22:59:15.016348 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:15.016979 24703 external_mini_cluster.cc:1428] Setting key 65cd537ea8b1d938ab3ece7b90997f19
I20260809 22:59:15.029760 30567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b: Bootstrap starting.
I20260809 22:59:15.031080 30570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:15.018971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.24.31.254:39651)
0809 22:59:15.020058 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:15.020069 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:15.020109 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:59:15.021948 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:15.021971 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:15.022430 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:15.022769 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:15.023486 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.024715 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.026722 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.027393 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.027543 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:15.028795 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:15.028802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:15.028815 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:15.029435 (+ 620us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:15.029522 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:15.029543 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:15.029627 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:15.029888 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 22:59:15.030157 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":420,"threads_started":1}
I20260809 22:59:15.033439 30567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:15.034699 30567 log.cc:824] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:15.035149 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:59:15.037681 30567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b: No bootstrap required, opened a new log
I20260809 22:59:15.045650 30567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53864b864d1846a1af7c16dc082fb91b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39651 } }
I20260809 22:59:15.045972 30567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:15.046047 30567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53864b864d1846a1af7c16dc082fb91b, State: Initialized, Role: FOLLOWER
I20260809 22:59:15.046514 30567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53864b864d1846a1af7c16dc082fb91b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39651 } }
I20260809 22:59:15.046701 30567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:15.046806 30567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:15.046918 30567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:15.049422 30567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53864b864d1846a1af7c16dc082fb91b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39651 } }
I20260809 22:59:15.049904 30567 leader_election.cc:302] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53864b864d1846a1af7c16dc082fb91b; no voters:
I20260809 22:59:15.050488 30567 leader_election.cc:288] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:15.050618 30572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:15.051575 30572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [term 1 LEADER]: Becoming Leader. State: Replica: 53864b864d1846a1af7c16dc082fb91b, State: Running, Role: LEADER
I20260809 22:59:15.052130 30572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53864b864d1846a1af7c16dc082fb91b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39651 } }
I20260809 22:59:15.052698 30567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:15.055200 30573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53864b864d1846a1af7c16dc082fb91b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53864b864d1846a1af7c16dc082fb91b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39651 } } }
I20260809 22:59:15.055593 30573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:15.059767 30574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53864b864d1846a1af7c16dc082fb91b. Latest consensus state: current_term: 1 leader_uuid: "53864b864d1846a1af7c16dc082fb91b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53864b864d1846a1af7c16dc082fb91b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39651 } } }
I20260809 22:59:15.060129 30574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:15.061426 30579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:15.067322 30579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:15.076459 30579 catalog_manager.cc:1347] Generated new cluster ID: 52bcee032842420f8f0c0cba0563cef3
I20260809 22:59:15.076622 30579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:15.098394 30579 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:15.099727 30579 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:15.106796 30579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53864b864d1846a1af7c16dc082fb91b: Generated new TSK 0
I20260809 22:59:15.107631 30579 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 22:59:15.170506 30602 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 22:59:15.170735 30602 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:59:15.171562 30570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:15.141768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.24.31.254:39651)
0809 22:59:15.142042 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:15.142049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:15.142501 (+ 452us) server_negotiation.cc:407] Connection header received
0809 22:59:15.142906 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:15.142910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:15.142981 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:15.143120 (+ 139us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:59:15.143786 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.144556 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.146755 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.147199 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.147296 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:15.147925 (+ 629us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:59:15.171002 (+ 23077us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 22:59:15.171314 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59760: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 22:59:15.171715 30570 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59760: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 22:59:15.173547 30601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:15.141638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39651 (local address 127.0.0.1:59760)
0809 22:59:15.142367 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:15.142389 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:15.142615 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:15.143181 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:15.143187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:15.143200 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:59:15.143633 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:15.143645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.144706 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.144713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:15.146617 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:15.146632 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.147385 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.147392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:15.147570 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:15.147652 (+ 82us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:59:15.171324 (+ 23672us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 22:59:15.171705 (+ 381us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.31.254:39651: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":354,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.31.254:39651: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 22:59:15.175293 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 30494
I20260809 22:59:15.176545 30561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:15.274703 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 30494
2026-08-09T22:59:15Z chronyd exiting
I20260809 22:59:15.330536 24703 test_util.cc:175] -----------------------------------------------
I20260809 22:59:15.330717 24703 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316311377261-24703-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (783 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 22:59:15.334606 24703 mini_oidc.cc:152] Starting JWKS server
I20260809 22:59:15.336508 24703 webserver.cc:460] Webserver started at https://127.0.0.1:40579/ using document root <none> and password file <none>
I20260809 22:59:15.336760 24703 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:59:15.337494 24703 webserver.cc:460] Webserver started at http://127.0.0.1:44299/ using document root <none> and password file <none>
2026-08-09T22:59:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:15Z Disabled control of system clock
I20260809 22:59:15.368054 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:33769
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:35057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:33769
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40579/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/testchainca.cert with env {}
W20260809 22:59:15.695498 30610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:15.695859 30610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:15.696027 30610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:15.703315 30610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:15.703483 30610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:15.703569 30610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:15.703681 30610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:15.703758 30610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:59:15.703846 30610 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:59:15.703922 30610 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40579/jwks.json
I20260809 22:59:15.714025 30610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:35057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:33769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:33769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40579/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:15.715379 30610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:15.716733 30610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:15.728706 30616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:15.728657 30618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:15.728752 30615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:15.729593 30610 server_base.cc:1034] running on GCE node
I20260809 22:59:15.730284 30610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:15.731565 30610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:15.732764 30610 hybrid_clock.cc:648] HybridClock initialized: now 1786316355732724 us; error 30 us; skew 500 ppm
I20260809 22:59:15.733121 30610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:59:15.735384 30610 webserver.cc:460] Webserver started at http://127.24.31.254:37441/ using document root <none> and password file <none>
I20260809 22:59:15.736070 30610 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:15.736191 30610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:15.736531 30610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:15.739530 30610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "2f2050ce280249b9bb6a9c50ab4f0a27"
format_stamp: "Formatted at 2026-08-09 22:59:15 on dist-test-slave-vxj2"
server_key: "ee1e73229f59a09ee0e5d192c184d0ec"
server_key_iv: "2d9366aec203a5a266975f0ac506f8ce"
server_key_version: "encryptionkey@0"
I20260809 22:59:15.740329 30610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "2f2050ce280249b9bb6a9c50ab4f0a27"
format_stamp: "Formatted at 2026-08-09 22:59:15 on dist-test-slave-vxj2"
server_key: "ee1e73229f59a09ee0e5d192c184d0ec"
server_key_iv: "2d9366aec203a5a266975f0ac506f8ce"
server_key_version: "encryptionkey@0"
I20260809 22:59:15.745874 30610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:59:15.749573 30624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:15.750916 30610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 22:59:15.751121 30610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "2f2050ce280249b9bb6a9c50ab4f0a27"
format_stamp: "Formatted at 2026-08-09 22:59:15 on dist-test-slave-vxj2"
server_key: "ee1e73229f59a09ee0e5d192c184d0ec"
server_key_iv: "2d9366aec203a5a266975f0ac506f8ce"
server_key_version: "encryptionkey@0"
I20260809 22:59:15.751363 30610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:15.782012 30610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:15.782915 30610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:15.783288 30610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:15.802942 30610 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:33769
I20260809 22:59:15.802965 30682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:33769 every 8 connection(s)
I20260809 22:59:15.804445 30610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:15.807168 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 30610
I20260809 22:59:15.807379 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:15.808058 24703 external_mini_cluster.cc:1428] Setting key c4345908b5738ab4cacffbb8ebaefac6
I20260809 22:59:15.808370 30683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:15.823493 30686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:15.810055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.24.31.254:33769)
0809 22:59:15.811155 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:15.811170 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:15.811225 (+ 55us) server_negotiation.cc:407] Connection header received
0809 22:59:15.813359 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:15.813391 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:15.813987 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:15.814404 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:15.814945 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.816941 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.818379 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.819419 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.819628 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:15.820752 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:15.820762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:15.820784 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:15.821523 (+ 739us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:15.821622 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:15.821653 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:15.821765 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:15.821982 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 22:59:15.822267 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":363,"threads_started":1}
I20260809 22:59:15.823767 30683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27: Bootstrap starting.
I20260809 22:59:15.827611 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:59:15.827598 30683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:15.828784 30683 log.cc:824] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:15.831840 30683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27: No bootstrap required, opened a new log
I20260809 22:59:15.838833 30683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2050ce280249b9bb6a9c50ab4f0a27" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33769 } }
I20260809 22:59:15.839154 30683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:15.839224 30683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2050ce280249b9bb6a9c50ab4f0a27, State: Initialized, Role: FOLLOWER
I20260809 22:59:15.839670 30683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2050ce280249b9bb6a9c50ab4f0a27" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33769 } }
I20260809 22:59:15.839867 30683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:15.839962 30683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:15.840078 30683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:15.842211 30683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2050ce280249b9bb6a9c50ab4f0a27" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33769 } }
I20260809 22:59:15.842793 30683 leader_election.cc:302] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f2050ce280249b9bb6a9c50ab4f0a27; no voters:
I20260809 22:59:15.843461 30683 leader_election.cc:288] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:15.843598 30688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:15.844449 30688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [term 1 LEADER]: Becoming Leader. State: Replica: 2f2050ce280249b9bb6a9c50ab4f0a27, State: Running, Role: LEADER
I20260809 22:59:15.844993 30688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2050ce280249b9bb6a9c50ab4f0a27" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33769 } }
I20260809 22:59:15.845623 30683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:15.847714 30690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f2050ce280249b9bb6a9c50ab4f0a27. Latest consensus state: current_term: 1 leader_uuid: "2f2050ce280249b9bb6a9c50ab4f0a27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2050ce280249b9bb6a9c50ab4f0a27" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33769 } } }
I20260809 22:59:15.848188 30690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:15.848654 30689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f2050ce280249b9bb6a9c50ab4f0a27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f2050ce280249b9bb6a9c50ab4f0a27" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33769 } } }
I20260809 22:59:15.849045 30689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:15.852017 30694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:15.857054 30694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:15.865847 30694 catalog_manager.cc:1347] Generated new cluster ID: 5c2d1cc4b99745c8bd204f46243b1382
I20260809 22:59:15.865984 30694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:15.901638 30694 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:15.902945 30694 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:15.908476 30694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f2050ce280249b9bb6a9c50ab4f0a27: Generated new TSK 0
I20260809 22:59:15.909404 30694 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 22:59:15.953063 30717 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 22:59:15.953226 30717 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:59:15.954566 30686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:15.934162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44570 (local address 127.24.31.254:33769)
0809 22:59:15.934356 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:15.934362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:15.934717 (+ 355us) server_negotiation.cc:407] Connection header received
0809 22:59:15.934935 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:15.934941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:15.935041 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:15.935215 (+ 174us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:59:15.935660 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.936823 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.938343 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.939094 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.939219 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:15.939708 (+ 489us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:59:15.953791 (+ 14083us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 22:59:15.954204 (+ 413us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44570: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 22:59:15.954708 30716 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:15.933966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:33769 (local address 127.0.0.1:44570)
0809 22:59:15.934605 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:15.934621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:15.934742 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:15.935235 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:15.935239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:15.935249 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:59:15.935497 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:15.935505 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.936991 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.936995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:15.938191 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:15.938209 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:15.939248 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:15.939252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:15.939394 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:15.939426 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:59:15.954054 (+ 14628us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 22:59:15.954254 (+ 200us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.31.254:33769: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":349,"threads_started":1}
W20260809 22:59:15.954842 30686 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44570: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40579/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260809 22:59:15.955964 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 30610
I20260809 22:59:15.957219 30678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:16.054224 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 30610
2026-08-09T22:59:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (775 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 22:59:16.109845 24703 mini_oidc.cc:152] Starting JWKS server
I20260809 22:59:16.111868 24703 webserver.cc:460] Webserver started at https://127.0.0.1:44803/ using document root <none> and password file <none>
I20260809 22:59:16.112131 24703 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 22:59:16.112744 24703 webserver.cc:460] Webserver started at http://127.0.0.1:46093/ using document root <none> and password file <none>
2026-08-09T22:59:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:16Z Disabled control of system clock
I20260809 22:59:16.143247 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42439
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:42439
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44803/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 22:59:16.470223 30724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:16.470611 30724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:16.470762 30724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:16.478309 30724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:16.478449 30724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:16.478538 30724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:16.478646 30724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:16.478727 30724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:59:16.478817 30724 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:59:16.478893 30724 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44803/jwks.json
I20260809 22:59:16.488121 30724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:42439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44803/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:16.489600 30724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:16.491456 30724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:16.503578 30729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:16.503674 30730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:16.503674 30732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:16.504554 30724 server_base.cc:1034] running on GCE node
I20260809 22:59:16.505491 30724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:16.506932 30724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:16.508232 30724 hybrid_clock.cc:648] HybridClock initialized: now 1786316356508186 us; error 29 us; skew 500 ppm
I20260809 22:59:16.508625 30724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:59:16.511123 30724 webserver.cc:460] Webserver started at http://127.24.31.254:45669/ using document root <none> and password file <none>
I20260809 22:59:16.511818 30724 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:16.511910 30724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:16.512283 30724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:16.515228 30724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "2871871ae19b43a8b66ab052a67b3264"
format_stamp: "Formatted at 2026-08-09 22:59:16 on dist-test-slave-vxj2"
server_key: "618d0df3b4ef02d670c9e3cb681bf274"
server_key_iv: "969dd7996216b127b4d1e511f5f79dad"
server_key_version: "encryptionkey@0"
I20260809 22:59:16.516036 30724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "2871871ae19b43a8b66ab052a67b3264"
format_stamp: "Formatted at 2026-08-09 22:59:16 on dist-test-slave-vxj2"
server_key: "618d0df3b4ef02d670c9e3cb681bf274"
server_key_iv: "969dd7996216b127b4d1e511f5f79dad"
server_key_version: "encryptionkey@0"
I20260809 22:59:16.521019 30724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260809 22:59:16.524355 30739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:16.525687 30724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:16.525884 30724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "2871871ae19b43a8b66ab052a67b3264"
format_stamp: "Formatted at 2026-08-09 22:59:16 on dist-test-slave-vxj2"
server_key: "618d0df3b4ef02d670c9e3cb681bf274"
server_key_iv: "969dd7996216b127b4d1e511f5f79dad"
server_key_version: "encryptionkey@0"
I20260809 22:59:16.526158 30724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:16.540390 30724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:16.541257 30724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:16.541574 30724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:16.560768 30724 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:42439
I20260809 22:59:16.560788 30796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:42439 every 8 connection(s)
I20260809 22:59:16.562187 30724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:16.565917 30798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:16.572629 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 30724
I20260809 22:59:16.572875 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:16.573607 24703 external_mini_cluster.cc:1428] Setting key 4ba727d99ec528fc5ae3c9e14231d85e
I20260809 22:59:16.576273 30798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264: Bootstrap starting.
I20260809 22:59:16.579300 30798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:16.580418 30798 log.cc:824] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:16.583145 30798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264: No bootstrap required, opened a new log
I20260809 22:59:16.586840 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:16.575469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45344 (local address 127.24.31.254:42439)
0809 22:59:16.576436 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:16.576451 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:16.576502 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:59:16.578246 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:16.578268 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:16.578702 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:16.579090 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:16.579638 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:16.581008 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:16.582370 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:16.583021 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:16.583232 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:16.584385 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:16.584392 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:16.584406 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:16.585037 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:16.585121 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:16.585145 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:16.585328 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:16.585573 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 22:59:16.585883 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":313,"threads_started":1}
I20260809 22:59:16.590763 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:59:16.591274 30798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871871ae19b43a8b66ab052a67b3264" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42439 } }
I20260809 22:59:16.591646 30798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:16.591755 30798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2871871ae19b43a8b66ab052a67b3264, State: Initialized, Role: FOLLOWER
I20260809 22:59:16.592392 30798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871871ae19b43a8b66ab052a67b3264" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42439 } }
I20260809 22:59:16.592638 30798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:16.592779 30798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:16.592953 30798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:16.595696 30798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871871ae19b43a8b66ab052a67b3264" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42439 } }
I20260809 22:59:16.596144 30798 leader_election.cc:302] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2871871ae19b43a8b66ab052a67b3264; no voters:
I20260809 22:59:16.596738 30798 leader_election.cc:288] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:16.596909 30803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:16.597764 30803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [term 1 LEADER]: Becoming Leader. State: Replica: 2871871ae19b43a8b66ab052a67b3264, State: Running, Role: LEADER
I20260809 22:59:16.598449 30803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871871ae19b43a8b66ab052a67b3264" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42439 } }
I20260809 22:59:16.598989 30798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:16.601203 30804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2871871ae19b43a8b66ab052a67b3264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871871ae19b43a8b66ab052a67b3264" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42439 } } }
I20260809 22:59:16.601616 30804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:16.602175 30805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2871871ae19b43a8b66ab052a67b3264. Latest consensus state: current_term: 1 leader_uuid: "2871871ae19b43a8b66ab052a67b3264" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2871871ae19b43a8b66ab052a67b3264" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42439 } } }
I20260809 22:59:16.602537 30805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:16.604595 30813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:16.610146 30813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:16.618450 30813 catalog_manager.cc:1347] Generated new cluster ID: d925ab612ef44905af9825fdf7bf54dc
I20260809 22:59:16.618571 30813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:16.635367 30813 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:16.636605 30813 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:16.648983 30813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2871871ae19b43a8b66ab052a67b3264: Generated new TSK 0
I20260809 22:59:16.649969 30813 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:59:16.674095 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:16.665509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45350 (local address 127.24.31.254:42439)
0809 22:59:16.665709 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:16.665717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:16.666156 (+ 439us) server_negotiation.cc:407] Connection header received
0809 22:59:16.666357 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:16.666363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:16.666458 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:16.666629 (+ 171us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:59:16.667061 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:16.668231 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:16.669656 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:16.670357 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:16.670485 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:16.670965 (+ 480us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:59:16.673401 (+ 2436us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44803/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 22:59:16.673747 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45350: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44803/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 22:59:16.674233 30801 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45350: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44803/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 22:59:16.675756 30833 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:16.665429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:42439 (local address 127.0.0.1:45350)
0809 22:59:16.666058 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:16.666074 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:16.666192 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:16.666645 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:16.666649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:16.666658 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:59:16.666919 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:16.666927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:16.668373 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:16.668376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:16.669521 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:16.669533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:16.670534 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:16.670538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:16.670663 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:16.670693 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:59:16.673604 (+ 2911us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44803/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 22:59:16.675321 (+ 1717us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.31.254:42439: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44803/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":1413,"thread_start_us":334,"threads_started":1}
I20260809 22:59:16.676789 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 30724
I20260809 22:59:16.678066 30790 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 22:59:16.689675 30834 webserver.cc:533] Webserver: sslize failed:
E20260809 22:59:16.689833 30834 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 22:59:16.773767 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 30724
2026-08-09T22:59:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (721 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T22:59:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:16Z Disabled control of system clock
I20260809 22:59:16.861217 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44725
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:44725
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/testchainca.cert with env {}
W20260809 22:59:17.186987 30841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:17.187286 30841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:17.187379 30841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:17.194679 30841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:17.194798 30841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:17.194836 30841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:17.194888 30841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:17.194933 30841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 22:59:17.195003 30841 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 22:59:17.195043 30841 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 22:59:17.205230 30841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:44725
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:17.206616 30841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:17.207902 30841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:17.221684 30846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:17.221640 30847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:17.221987 30849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:17.222656 30841 server_base.cc:1034] running on GCE node
I20260809 22:59:17.223428 30841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:17.224821 30841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:17.226052 30841 hybrid_clock.cc:648] HybridClock initialized: now 1786316357226012 us; error 41 us; skew 500 ppm
I20260809 22:59:17.226830 30841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 22:59:17.229261 30841 webserver.cc:460] Webserver started at http://127.24.31.254:34663/ using document root <none> and password file <none>
I20260809 22:59:17.230046 30841 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:17.230176 30841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:17.230511 30841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:17.233479 30841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f"
format_stamp: "Formatted at 2026-08-09 22:59:17 on dist-test-slave-vxj2"
server_key: "48c6da66f4c0390d749232be59327988"
server_key_iv: "300f6e91cbca777c1a8c3c1a356e0ece"
server_key_version: "encryptionkey@0"
I20260809 22:59:17.234308 30841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f"
format_stamp: "Formatted at 2026-08-09 22:59:17 on dist-test-slave-vxj2"
server_key: "48c6da66f4c0390d749232be59327988"
server_key_iv: "300f6e91cbca777c1a8c3c1a356e0ece"
server_key_version: "encryptionkey@0"
I20260809 22:59:17.239432 30841 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:59:17.242877 30856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:17.244395 30841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:59:17.244613 30841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f"
format_stamp: "Formatted at 2026-08-09 22:59:17 on dist-test-slave-vxj2"
server_key: "48c6da66f4c0390d749232be59327988"
server_key_iv: "300f6e91cbca777c1a8c3c1a356e0ece"
server_key_version: "encryptionkey@0"
I20260809 22:59:17.244853 30841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:17.256558 30841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:17.257458 30841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:17.257789 30841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:17.277231 30841 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:44725
I20260809 22:59:17.277256 30913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:44725 every 8 connection(s)
I20260809 22:59:17.278653 30841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:17.280234 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 30841
I20260809 22:59:17.280436 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:17.281356 24703 external_mini_cluster.cc:1428] Setting key 62ecf04cdeea13275eb81894731853a2
I20260809 22:59:17.283546 30914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:17.297236 30918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:17.283313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.24.31.254:44725)
0809 22:59:17.284339 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:17.284354 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:17.284405 (+ 51us) server_negotiation.cc:407] Connection header received
0809 22:59:17.286603 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:17.286631 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:17.287230 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:17.287672 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:17.288207 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:17.290353 (+ 2146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:17.291757 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:17.292784 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:17.292972 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:17.294309 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:17.294320 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:17.294338 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 22:59:17.295093 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:17.295197 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 22:59:17.295223 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:17.295357 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:17.295586 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 22:59:17.295928 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":331,"threads_started":1}
I20260809 22:59:17.298245 30914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f: Bootstrap starting.
I20260809 22:59:17.301469 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 22:59:17.302132 30914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:17.303337 30914 log.cc:824] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:17.306594 30914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f: No bootstrap required, opened a new log
I20260809 22:59:17.314769 30914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44725 } }
I20260809 22:59:17.315115 30914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:17.315191 30914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff392b5153e4e45bbf13f6d9c3ab00f, State: Initialized, Role: FOLLOWER
I20260809 22:59:17.315775 30914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44725 } }
I20260809 22:59:17.315984 30914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:17.316082 30914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:17.316249 30914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:17.318765 30914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44725 } }
I20260809 22:59:17.319218 30914 leader_election.cc:302] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ff392b5153e4e45bbf13f6d9c3ab00f; no voters:
I20260809 22:59:17.319783 30914 leader_election.cc:288] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:17.319932 30920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:17.320650 30920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [term 1 LEADER]: Becoming Leader. State: Replica: 6ff392b5153e4e45bbf13f6d9c3ab00f, State: Running, Role: LEADER
I20260809 22:59:17.321233 30920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44725 } }
I20260809 22:59:17.321965 30914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:17.323868 30922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ff392b5153e4e45bbf13f6d9c3ab00f. Latest consensus state: current_term: 1 leader_uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44725 } } }
I20260809 22:59:17.324244 30922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:17.325254 30921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ff392b5153e4e45bbf13f6d9c3ab00f" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44725 } } }
I20260809 22:59:17.325595 30921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:17.326530 30927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:17.331708 30927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:17.340700 30927 catalog_manager.cc:1347] Generated new cluster ID: c16ec8be7378477f958ba095d72bf798
I20260809 22:59:17.340804 30927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:17.349663 30927 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:17.351432 30927 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:17.357903 30927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ff392b5153e4e45bbf13f6d9c3ab00f: Generated new TSK 0
I20260809 22:59:17.358860 30927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:59:17.439599 30918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:17.376813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.24.31.254:44725)
0809 22:59:17.377032 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:17.377042 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:17.377557 (+ 515us) server_negotiation.cc:407] Connection header received
0809 22:59:17.377793 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:17.377797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:17.377871 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:17.377997 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0809 22:59:17.378526 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:17.379186 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:17.381192 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:17.382059 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:17.382202 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:17.382971 (+ 769us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 22:59:17.438997 (+ 56026us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 22:59:17.439336 (+ 339us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60168: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 22:59:17.439781 30918 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60168: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 22:59:17.440073 30945 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:17.376789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44725 (local address 127.0.0.1:60168)
0809 22:59:17.377449 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:17.377469 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:17.377614 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:17.378046 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:17.378051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:17.378064 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 22:59:17.378381 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:17.378394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:17.379339 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:17.379345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:17.381005 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:17.381020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:17.382338 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:17.382345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:17.382528 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:17.382576 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 22:59:17.439314 (+ 56738us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 22:59:17.439644 (+ 330us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.31.254:44725: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":383,"threads_started":1}
I20260809 22:59:17.441638 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 30841
I20260809 22:59:17.442889 30909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:17.542464 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 30841
2026-08-09T22:59:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (748 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (423 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (420 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:59:18 dist-test-slave-vxj2 krb5kdc[30961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:59:18 dist-test-slave-vxj2 krb5kdc[30961](info): set up 2 sockets
Aug 09 22:59:18 dist-test-slave-vxj2 krb5kdc[30961](info): commencing operation
krb5kdc: starting...
W20260809 22:59:19.555728 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:59:19 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:59:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:59:19.789705 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:37333
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:37333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:20.129009 30976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:20.129385 30976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:20.129563 30976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:20.136888 30976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:20.137001 30976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:20.137049 30976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:20.137138 30976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:20.137225 30976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:20.147310 30976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:37333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:37333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:20.148681 30976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:20.150082 30976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:20.162460 30983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:20.162511 30985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:20.162541 30982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:20.166659 30976 server_base.cc:1034] running on GCE node
I20260809 22:59:20.167459 30976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:20.168922 30976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:20.170123 30976 hybrid_clock.cc:648] HybridClock initialized: now 1786316360170051 us; error 64 us; skew 500 ppm
I20260809 22:59:20.170522 30976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:20 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316360, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:20.178017 30976 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:59:20.179590 30976 webserver.cc:460] Webserver started at http://127.24.31.254:37425/ using document root <none> and password file <none>
I20260809 22:59:20.180276 30976 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:20.180456 30976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:20.180756 30976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:20.183773 30976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8"
format_stamp: "Formatted at 2026-08-09 22:59:20 on dist-test-slave-vxj2"
server_key: "1c8b91a35d72304b4210e14930e514da"
server_key_iv: "4447dd7748bbc8abc117f39d4aebba39"
server_key_version: "encryptionkey@0"
I20260809 22:59:20.184547 30976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8"
format_stamp: "Formatted at 2026-08-09 22:59:20 on dist-test-slave-vxj2"
server_key: "1c8b91a35d72304b4210e14930e514da"
server_key_iv: "4447dd7748bbc8abc117f39d4aebba39"
server_key_version: "encryptionkey@0"
I20260809 22:59:20.189652 30976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:59:20.193192 30993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:20.194505 30976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:59:20.194687 30976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8"
format_stamp: "Formatted at 2026-08-09 22:59:20 on dist-test-slave-vxj2"
server_key: "1c8b91a35d72304b4210e14930e514da"
server_key_iv: "4447dd7748bbc8abc117f39d4aebba39"
server_key_version: "encryptionkey@0"
I20260809 22:59:20.194882 30976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:20.216277 30976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:20.220726 30976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:20.221077 30976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:20.240582 30976 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:37333
I20260809 22:59:20.240633 31049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:37333 every 8 connection(s)
I20260809 22:59:20.242094 30976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:20.246088 31050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:20.251930 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 30976
I20260809 22:59:20.252161 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:20.252983 24703 external_mini_cluster.cc:1428] Setting key 36a1bb8977581a61683acb631acf3ef0
I20260809 22:59:20.257041 31050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8: Bootstrap starting.
I20260809 22:59:20.260744 31050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:20.261857 31050 log.cc:824] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:20.264578 31050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8: No bootstrap required, opened a new log
Aug 09 22:59:20 dist-test-slave-vxj2 krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:20.272569 31050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37333 } }
I20260809 22:59:20.272944 31050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:20.273047 31050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b962acdd3d0417abf8e1b9dbefbdaa8, State: Initialized, Role: FOLLOWER
I20260809 22:59:20.273682 31050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37333 } }
I20260809 22:59:20.273962 31050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:20.274148 31050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:20.274336 31050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:20.276034 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:20.255207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.24.31.254:37333)
0809 22:59:20.256045 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:20.256056 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:20.256100 (+ 44us) server_negotiation.cc:407] Connection header received
0809 22:59:20.257920 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:20.257942 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:20.258311 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:20.258634 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:20.260467 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:20.261868 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:20.263549 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:20.264195 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:20.264396 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:20.268611 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:20.268618 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:20.268635 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:20.268671 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:20.271863 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:20.272582 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:20.272588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:20.272594 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:20.272751 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:20.273183 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:20.273189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:20.273192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:20.273830 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:20.274145 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:20.274597 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 22:59:20.275028 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":424,"threads_started":1}
I20260809 22:59:20.276872 31050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37333 } }
I20260809 22:59:20.277433 31050 leader_election.cc:302] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b962acdd3d0417abf8e1b9dbefbdaa8; no voters:
I20260809 22:59:20.278096 31050 leader_election.cc:288] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:20.278218 31055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:20.279340 31055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [term 1 LEADER]: Becoming Leader. State: Replica: 3b962acdd3d0417abf8e1b9dbefbdaa8, State: Running, Role: LEADER
I20260809 22:59:20.279903 31055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37333 } }
I20260809 22:59:20.280395 31050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:20.282825 31057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b962acdd3d0417abf8e1b9dbefbdaa8. Latest consensus state: current_term: 1 leader_uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37333 } } }
I20260809 22:59:20.282927 31056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b962acdd3d0417abf8e1b9dbefbdaa8" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37333 } } }
I20260809 22:59:20.283277 31057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:20.283349 31056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:20.284708 31060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:20.289557 31060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:20.298013 31060 catalog_manager.cc:1347] Generated new cluster ID: 6620d4af1cf4492f86c985d442e10ddb
I20260809 22:59:20.298122 31060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:20.309048 31060 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:20.310292 31060 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:20.319191 31060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b962acdd3d0417abf8e1b9dbefbdaa8: Generated new TSK 0
I20260809 22:59:20.320061 31060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:59:20.378471 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37333
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:20.710815 31078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:20.711099 31078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:20.711277 31078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:20.718417 31078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:20.718519 31078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:20.718719 31078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:20.728304 31078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:20.729631 31078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:20.730988 31078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:20.744722 31084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:20.744647 31086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:20.744683 31088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:20.745461 31078 server_base.cc:1034] running on GCE node
I20260809 22:59:20.746013 31078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:20.746915 31078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:20.748121 31078 hybrid_clock.cc:648] HybridClock initialized: now 1786316360748084 us; error 35 us; skew 500 ppm
I20260809 22:59:20.748402 31078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:20 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316360, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:20.751665 31078 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:59:20.753044 31078 webserver.cc:460] Webserver started at http://127.24.31.193:41281/ using document root <none> and password file <none>
I20260809 22:59:20.753851 31078 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:20.753957 31078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:20.754211 31078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:20.757038 31078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "b3d30f6066d04ad4800da038d2c59684"
format_stamp: "Formatted at 2026-08-09 22:59:20 on dist-test-slave-vxj2"
server_key: "2604620c988af41fffe3c7d9bf998cf9"
server_key_iv: "3b883b4144e311df35304a75e39fbe1b"
server_key_version: "encryptionkey@0"
I20260809 22:59:20.757966 31078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "b3d30f6066d04ad4800da038d2c59684"
format_stamp: "Formatted at 2026-08-09 22:59:20 on dist-test-slave-vxj2"
server_key: "2604620c988af41fffe3c7d9bf998cf9"
server_key_iv: "3b883b4144e311df35304a75e39fbe1b"
server_key_version: "encryptionkey@0"
I20260809 22:59:20.762938 31078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:59:20.766185 31095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:20.767439 31078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:20.767603 31078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "b3d30f6066d04ad4800da038d2c59684"
format_stamp: "Formatted at 2026-08-09 22:59:20 on dist-test-slave-vxj2"
server_key: "2604620c988af41fffe3c7d9bf998cf9"
server_key_iv: "3b883b4144e311df35304a75e39fbe1b"
server_key_version: "encryptionkey@0"
I20260809 22:59:20.767809 31078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:20.780562 31078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:20.784497 31078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:20.784775 31078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:20.785717 31078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:20.786983 31078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:20.787055 31078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:20.787218 31078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:20.787302 31078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:20.821262 31078 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:40909
I20260809 22:59:20.821362 31218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:40909 every 8 connection(s)
I20260809 22:59:20.822566 31078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:20.830649 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 31078
I20260809 22:59:20.830874 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:59:20.831554 24703 external_mini_cluster.cc:1428] Setting key 0c2e4826b2a0de35d5c9edf395b3a6d3
Aug 09 22:59:20 dist-test-slave-vxj2 krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316360, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:20.844048 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:20.825292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:41693 (local address 127.24.31.254:37333)
0809 22:59:20.825746 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:20.825751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:20.827016 (+ 1265us) server_negotiation.cc:407] Connection header received
0809 22:59:20.828797 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:20.828803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:20.828907 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:20.829032 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:20.830523 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:20.831353 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:20.833097 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:20.833728 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:20.833887 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:20.837244 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:20.837252 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:20.837260 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:20.837311 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:20.840858 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:20.841663 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:20.841670 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:20.841674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:20.841828 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:20.842351 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:20.842354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:20.842356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:20.842614 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:20.842726 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:20.843622 (+ 896us) server_negotiation.cc:299] Negotiation successful
0809 22:59:20.843805 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":239,"threads_started":1}
I20260809 22:59:20.845527 31222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:20.825862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:37333 (local address 127.24.31.193:41693)
0809 22:59:20.826856 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:20.826898 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:20.828530 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:20.829290 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:20.829298 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:20.829793 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:20.830339 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:20.830356 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:20.831534 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:20.831538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:20.832919 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:20.832934 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:20.833969 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:20.833974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:20.834164 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:20.834845 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:20.834867 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:20.836999 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:20.841007 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:20.841015 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:20.841027 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:20.841543 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:20.841945 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:20.841950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:20.841953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:20.842213 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:20.842747 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:20.842757 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:20.843207 (+ 450us) client_negotiation.cc:769] Sending connection context
0809 22:59:20.843511 (+ 304us) client_negotiation.cc:240] Negotiation successful
0809 22:59:20.844084 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":299,"threads_started":1}
I20260809 22:59:20.847123 31219 heartbeater.cc:347] Connected to a master server at 127.24.31.254:37333
I20260809 22:59:20.847486 31219 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:20.848253 31219 heartbeater.cc:511] Master 127.24.31.254:37333 requested a full tablet report, sending...
I20260809 22:59:20.850342 31013 ts_manager.cc:194] Registered new tserver with Master: b3d30f6066d04ad4800da038d2c59684 (127.24.31.193:40909)
I20260809 22:59:20.851857 31013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:41693
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:59:20.884506 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37333
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:21.207726 31226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:21.208067 31226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:21.208247 31226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:21.215750 31226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:21.215904 31226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:21.216084 31226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:21.226044 31226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:21.227380 31226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:21.228725 31226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:21.241927 31233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:21.242175 31232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:21.241930 31237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:21.243156 31226 server_base.cc:1034] running on GCE node
I20260809 22:59:21.243849 31226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:21.244966 31226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:21.246206 31226 hybrid_clock.cc:648] HybridClock initialized: now 1786316361246150 us; error 46 us; skew 500 ppm
I20260809 22:59:21.246603 31226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:21.250315 31226 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:59:21.251904 31226 webserver.cc:460] Webserver started at http://127.24.31.194:38859/ using document root <none> and password file <none>
I20260809 22:59:21.252621 31226 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:21.252772 31226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:21.253095 31226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:21.256168 31226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "7a6457c9903547feb94da27d82832fb2"
format_stamp: "Formatted at 2026-08-09 22:59:21 on dist-test-slave-vxj2"
server_key: "3ee3f908cb91471fafb0c7dfb5e093d9"
server_key_iv: "a959db180f0ff6ee4d34dfcc597c2ffc"
server_key_version: "encryptionkey@0"
I20260809 22:59:21.256989 31226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "7a6457c9903547feb94da27d82832fb2"
format_stamp: "Formatted at 2026-08-09 22:59:21 on dist-test-slave-vxj2"
server_key: "3ee3f908cb91471fafb0c7dfb5e093d9"
server_key_iv: "a959db180f0ff6ee4d34dfcc597c2ffc"
server_key_version: "encryptionkey@0"
I20260809 22:59:21.262282 31226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 22:59:21.265939 31244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:21.267262 31226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:59:21.267446 31226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "7a6457c9903547feb94da27d82832fb2"
format_stamp: "Formatted at 2026-08-09 22:59:21 on dist-test-slave-vxj2"
server_key: "3ee3f908cb91471fafb0c7dfb5e093d9"
server_key_iv: "a959db180f0ff6ee4d34dfcc597c2ffc"
server_key_version: "encryptionkey@0"
I20260809 22:59:21.267668 31226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:21.286679 31226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:21.291283 31226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:21.291581 31226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:21.292554 31226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:21.294008 31226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:21.294111 31226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:21.294265 31226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:21.294340 31226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:21.330444 31226 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:46139
I20260809 22:59:21.330503 31371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:46139 every 8 connection(s)
I20260809 22:59:21.331760 31226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:21.335969 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 31226
I20260809 22:59:21.336179 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:59:21.336799 24703 external_mini_cluster.cc:1428] Setting key 14c9d322e1bb6d35859aedf59fcab9f3
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:21.357342 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:21.334421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:37701 (local address 127.24.31.254:37333)
0809 22:59:21.334607 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:21.334612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:21.336054 (+ 1442us) server_negotiation.cc:407] Connection header received
0809 22:59:21.338622 (+ 2568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:21.338626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:21.338726 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:21.338844 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:21.341086 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.341849 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.344564 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.344990 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.345087 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:21.349847 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:21.349857 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:21.349864 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:21.349909 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:21.353731 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.354509 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.354518 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.354524 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.354697 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.355165 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.355173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.355179 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.355603 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:21.355762 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:21.356567 (+ 805us) server_negotiation.cc:299] Negotiation successful
0809 22:59:21.356874 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 22:59:21.357954 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:21.334888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:37333 (local address 127.24.31.194:37701)
0809 22:59:21.335837 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:21.335903 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:21.338312 (+ 2409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:21.339147 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:21.339156 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:21.339990 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:21.340869 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:21.340900 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.342029 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.342038 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:21.344401 (+ 2363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:21.344421 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.345271 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.345282 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:21.345647 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:21.346440 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:21.346490 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:21.349598 (+ 3108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:21.353906 (+ 4308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:21.353913 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:21.353926 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:21.354359 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:21.354845 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:21.354848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:21.354851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:21.355038 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:21.355728 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:21.355734 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:21.356085 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 22:59:21.356328 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 22:59:21.356734 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":316,"threads_started":1}
I20260809 22:59:21.359618 31373 heartbeater.cc:347] Connected to a master server at 127.24.31.254:37333
I20260809 22:59:21.359974 31373 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:21.360971 31373 heartbeater.cc:511] Master 127.24.31.254:37333 requested a full tablet report, sending...
I20260809 22:59:21.362735 31013 ts_manager.cc:194] Registered new tserver with Master: 7a6457c9903547feb94da27d82832fb2 (127.24.31.194:46139)
I20260809 22:59:21.363641 31013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:37701
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:59:21.396534 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37333
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:35211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:21.722719 31378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:21.723047 31378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:21.723145 31378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:21.730199 31378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:21.730304 31378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:21.730420 31378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:21.739801 31378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:35211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:37333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:21.740986 31378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:21.742246 31378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:21.754936 31383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:21.754885 31386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:21.754974 31384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:21.755761 31378 server_base.cc:1034] running on GCE node
I20260809 22:59:21.756320 31378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:21.757249 31378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:21.758478 31378 hybrid_clock.cc:648] HybridClock initialized: now 1786316361758439 us; error 57 us; skew 500 ppm
I20260809 22:59:21.758800 31378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:21.761909 31378 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:59:21.763325 31378 webserver.cc:460] Webserver started at http://127.24.31.195:34715/ using document root <none> and password file <none>
I20260809 22:59:21.763974 31378 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:21.764114 31378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:21.764449 31378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:21.767379 31378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "e3dfd6794e07498f9f91f132b4e21a8e"
format_stamp: "Formatted at 2026-08-09 22:59:21 on dist-test-slave-vxj2"
server_key: "b952ebefac71a5d997ac8dc7bea1b486"
server_key_iv: "690b1053faaf37a776fc45d06d39ad04"
server_key_version: "encryptionkey@0"
I20260809 22:59:21.768162 31378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "e3dfd6794e07498f9f91f132b4e21a8e"
format_stamp: "Formatted at 2026-08-09 22:59:21 on dist-test-slave-vxj2"
server_key: "b952ebefac71a5d997ac8dc7bea1b486"
server_key_iv: "690b1053faaf37a776fc45d06d39ad04"
server_key_version: "encryptionkey@0"
I20260809 22:59:21.773284 31378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 22:59:21.776484 31397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:21.777740 31378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 22:59:21.777912 31378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "e3dfd6794e07498f9f91f132b4e21a8e"
format_stamp: "Formatted at 2026-08-09 22:59:21 on dist-test-slave-vxj2"
server_key: "b952ebefac71a5d997ac8dc7bea1b486"
server_key_iv: "690b1053faaf37a776fc45d06d39ad04"
server_key_version: "encryptionkey@0"
I20260809 22:59:21.778097 31378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:21.791801 31378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:21.795452 31378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:21.795727 31378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:21.796679 31378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:21.797976 31378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:21.798072 31378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:21.798209 31378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:21.798282 31378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:21.834808 31378 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:37709
I20260809 22:59:21.834844 31518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:37709 every 8 connection(s)
I20260809 22:59:21.836109 31378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:21.837335 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 31378
I20260809 22:59:21.837531 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:59:21.838238 24703 external_mini_cluster.cc:1428] Setting key 9378c1c5865b8ff3bd86a7ed948b9eac
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:21.855016 31219 heartbeater.cc:503] Master 127.24.31.254:37333 was elected leader, sending a full tablet report...
I20260809 22:59:21.856755 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:21.838932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:55577 (local address 127.24.31.254:37333)
0809 22:59:21.839135 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:21.839142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:21.840301 (+ 1159us) server_negotiation.cc:407] Connection header received
0809 22:59:21.842444 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:21.842449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:21.842513 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:21.842593 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:21.843936 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.844709 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.846293 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.846699 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.846796 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:21.850168 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:21.850172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:21.850175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:21.850207 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:21.853373 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.854238 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.854241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.854243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.854376 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.854801 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.854807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.854810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.855187 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:21.855372 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:21.856178 (+ 806us) server_negotiation.cc:299] Negotiation successful
0809 22:59:21.856404 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:59:21.857604 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:21.839305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:37333 (local address 127.24.31.195:55577)
0809 22:59:21.840124 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:21.840158 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:21.842193 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:21.842780 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:21.842787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:21.843246 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:21.843798 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:21.843815 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.844837 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.844841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:21.846171 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:21.846183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.846931 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.846937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:21.847143 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:21.847809 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:21.847831 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:21.849984 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:21.853526 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:21.853533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:21.853544 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:21.854119 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:21.854471 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:21.854474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:21.854477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:21.854647 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:21.855324 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:21.855329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:21.855784 (+ 455us) client_negotiation.cc:769] Sending connection context
0809 22:59:21.856016 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 22:59:21.856358 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":420,"threads_started":1}
I20260809 22:59:21.858929 31519 heartbeater.cc:347] Connected to a master server at 127.24.31.254:37333
I20260809 22:59:21.859237 31519 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:21.859936 31519 heartbeater.cc:511] Master 127.24.31.254:37333 requested a full tablet report, sending...
I20260809 22:59:21.861304 31013 ts_manager.cc:194] Registered new tserver with Master: e3dfd6794e07498f9f91f132b4e21a8e (127.24.31.195:37709)
I20260809 22:59:21.862028 31013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:55577
I20260809 22:59:21.865913 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/test-admin.keytab.
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:21.901007 24703 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 22:59:21.908077 31527 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:21.904777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.24.31.193:40909)
0809 22:59:21.905503 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:21.905514 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:21.905550 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:59:21.905651 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:21.905661 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:21.906008 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:21.906182 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:21.907613 (+ 1431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53090: BlockingRecv error: recv got EOF from 127.0.0.1:53090 (error 108)
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":397,"threads_started":1}
W20260809 22:59:21.908118 24703 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5cc' of type FILE
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:21.910401 24703 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:21.912070 24703 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:59:21.930696 31527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:21.913816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53092 (local address 127.24.31.193:40909)
0809 22:59:21.914032 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:21.914039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:21.914377 (+ 338us) server_negotiation.cc:407] Connection header received
0809 22:59:21.914585 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:21.914593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:21.914670 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:21.914761 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:21.915824 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.917217 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.918895 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.919961 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.920123 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:21.924735 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:21.924749 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:21.924764 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:21.924796 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:21.927847 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.928500 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.928507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.928509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.928629 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.929045 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.929050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.929054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.929516 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:21.929819 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:21.930159 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 22:59:21.930433 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 22:59:21.936497 31527 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:21.934903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53102 (local address 127.24.31.193:40909)
0809 22:59:21.935105 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:21.935110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:21.935266 (+ 156us) server_negotiation.cc:407] Connection header received
0809 22:59:21.935385 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:21.935388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:21.935460 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:21.935537 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:21.936261 (+ 724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53102: BlockingRecv error: recv got EOF from 127.0.0.1:53102 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 22:59:21.936690 24703 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5cc' of type FILE
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:21.938265 24703 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:21.939627 24703 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:59:21.952976 31527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:21.940929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53118 (local address 127.24.31.193:40909)
0809 22:59:21.941076 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:21.941080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:21.941521 (+ 441us) server_negotiation.cc:407] Connection header received
0809 22:59:21.941618 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:21.941621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:21.941673 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:21.941749 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:21.942659 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.943379 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.944515 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.944930 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.945014 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:21.948288 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:21.948292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:21.948296 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:21.948337 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:21.950974 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.951536 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.951540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.951542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.951650 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.951960 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.951963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.951965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.952216 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:21.952418 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:21.952622 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 22:59:21.952769 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 22:59:21.957247 31527 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:21.955438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53130 (local address 127.24.31.193:40909)
0809 22:59:21.955606 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:21.955614 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:21.955983 (+ 369us) server_negotiation.cc:407] Connection header received
0809 22:59:21.956162 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:21.956167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:21.956240 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:21.956346 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:21.957009 (+ 663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53130: BlockingRecv error: recv got EOF from 127.0.0.1:53130 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 22:59:21.957773 24703 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316311377261-24703-0/krb5kdc/krb5cc)
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:21.959232 24703 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:21.960534 24703 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 22:59:21 dist-test-slave-vxj2 krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:59:21.974777 31527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:21.961733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.24.31.193:40909)
0809 22:59:21.961918 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:21.961925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:21.962221 (+ 296us) server_negotiation.cc:407] Connection header received
0809 22:59:21.962379 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:21.962384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:21.962464 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:21.962559 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:21.963469 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.964160 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.965407 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.965835 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.965964 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:21.969234 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:21.969245 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:21.969248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:21.969281 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:21.972080 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.972833 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.972840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.972843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.972996 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.973428 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.973433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.973437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.973711 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:21.974023 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:21.974290 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 22:59:21.974486 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:59:21.976154 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 31078
I20260809 22:59:21.987219 31527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:21.976889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53152 (local address 127.24.31.193:40909)
0809 22:59:21.977032 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:21.977035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:21.977347 (+ 312us) server_negotiation.cc:407] Connection header received
0809 22:59:21.977462 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:21.977465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:21.977510 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:21.977579 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:21.978356 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.978969 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.980075 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:21.980489 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:21.980617 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:21.982737 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:21.982742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:21.982745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:21.982772 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:21.985203 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.985729 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.985732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.985734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.985854 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:21.986169 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:21.986172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:21.986174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:21.986399 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:21.986596 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:21.986845 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 22:59:21.986986 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 22:59:21.987784 31214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:22.071522 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 31078
I20260809 22:59:22.090780 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 31226
Aug 09 22:59:22 dist-test-slave-vxj2 krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 22:59:22.106436 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:22.091642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58790 (local address 127.24.31.194:46139)
0809 22:59:22.092224 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:22.092230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:22.092256 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:59:22.092326 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:22.092332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:22.092539 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:22.092672 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:22.093722 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:22.094746 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:22.096028 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:22.096560 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:22.096655 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:22.100639 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:22.100651 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:22.100665 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:22.100697 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:22.103715 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:22.104378 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:22.104384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:22.104387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:22.104529 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:22.104947 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:22.104950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:22.104953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:22.105387 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:22.105637 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:22.105938 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 22:59:22.106141 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":336,"threads_started":1}
I20260809 22:59:22.107798 31366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:22.190231 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 31226
I20260809 22:59:22.208318 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 31378
Aug 09 22:59:22 dist-test-slave-vxj2 krb5kdc[30961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 22:59:22.224050 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:22.209464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.24.31.195:37709)
0809 22:59:22.210143 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:22.210150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:22.210179 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:59:22.210240 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:22.210246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:22.210478 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:22.210603 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:22.211594 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:22.212614 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:22.213937 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:22.214491 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:22.214590 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:22.218556 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:22.218568 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:22.218582 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:22.218613 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:22.221614 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:22.222253 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:22.222271 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:22.222273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:22.222390 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:22.222733 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:22.222736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:22.222738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:22.223065 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:22.223343 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:22.223553 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 22:59:22.223745 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":370,"threads_started":1}
I20260809 22:59:22.225409 31514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:22.314639 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 31378
I20260809 22:59:22.333444 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 30976
I20260809 22:59:22.334712 31045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:22.439760 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 30976
2026-08-09T22:59:22Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4060 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:59:22 dist-test-slave-vxj2 krb5kdc[31549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:59:22 dist-test-slave-vxj2 krb5kdc[31549](info): set up 2 sockets
Aug 09 22:59:22 dist-test-slave-vxj2 krb5kdc[31549](info): commencing operation
krb5kdc: starting...
W20260809 22:59:23.611202 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:59:23 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:59:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.254.keytab.
Entry for principal oryx/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.254.keytab.
I20260809 22:59:23.840612 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:46425
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.254.keytab
--principal=oryx/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:46425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:24.163827 31562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:24.164147 31562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:24.164320 31562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:24.171486 31562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:24.171622 31562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:24.171734 31562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:24.171838 31562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:24.171926 31562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:24.181223 31562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:46425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.254.keytab
--principal=oryx/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:46425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:24.182482 31562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:24.183714 31562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:24.195634 31567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:24.195679 31568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:24.195724 31571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:24.196863 31562 server_base.cc:1034] running on GCE node
I20260809 22:59:24.197660 31562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:24.199088 31562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:24.200331 31562 hybrid_clock.cc:648] HybridClock initialized: now 1786316364200297 us; error 54 us; skew 500 ppm
I20260809 22:59:24.200673 31562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:24 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316364, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:24.203781 31562 init.cc:377] Logged in from keytab as oryx/127.24.31.254@KRBTEST.COM (short username oryx)
I20260809 22:59:24.205288 31562 webserver.cc:460] Webserver started at http://127.24.31.254:37711/ using document root <none> and password file <none>
I20260809 22:59:24.205947 31562 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:24.206092 31562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:24.206400 31562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:24.209338 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "66318d8fd4694039bb253dc9c4f3640e"
format_stamp: "Formatted at 2026-08-09 22:59:24 on dist-test-slave-vxj2"
server_key: "3b0b0d180e8146ee8ff8fdb11ad0dab1"
server_key_iv: "e5840a945bbdde17b7d40c8a6ad10353"
server_key_version: "encryptionkey@0"
I20260809 22:59:24.210093 31562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "66318d8fd4694039bb253dc9c4f3640e"
format_stamp: "Formatted at 2026-08-09 22:59:24 on dist-test-slave-vxj2"
server_key: "3b0b0d180e8146ee8ff8fdb11ad0dab1"
server_key_iv: "e5840a945bbdde17b7d40c8a6ad10353"
server_key_version: "encryptionkey@0"
I20260809 22:59:24.215065 31562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:59:24.218340 31579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:24.219715 31562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:24.219915 31562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "66318d8fd4694039bb253dc9c4f3640e"
format_stamp: "Formatted at 2026-08-09 22:59:24 on dist-test-slave-vxj2"
server_key: "3b0b0d180e8146ee8ff8fdb11ad0dab1"
server_key_iv: "e5840a945bbdde17b7d40c8a6ad10353"
server_key_version: "encryptionkey@0"
I20260809 22:59:24.220108 31562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:24.237353 31562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:24.241736 31562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:24.242056 31562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:24.261361 31562 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:46425
I20260809 22:59:24.261358 31634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:46425 every 8 connection(s)
I20260809 22:59:24.263136 31562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:24.268186 31635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:24.272482 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 31562
I20260809 22:59:24.272701 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:24.273469 24703 external_mini_cluster.cc:1428] Setting key 1121273224ab6cc4a5d2d79b30faf09b
I20260809 22:59:24.281214 31635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: Bootstrap starting.
I20260809 22:59:24.284793 31635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:24.285964 31635 log.cc:824] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:24.288566 31635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: No bootstrap required, opened a new log
Aug 09 22:59:24 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:24.299007 31639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:24.275555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49296 (local address 127.24.31.254:46425)
0809 22:59:24.277775 (+ 2220us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:24.277787 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:24.277827 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:59:24.279634 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:24.279656 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:24.280025 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:24.280350 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:24.282297 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:24.283627 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:24.285441 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:24.286152 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:24.286275 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:24.290318 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:24.290328 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:24.290351 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:24.290393 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:24.294541 (+ 4148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:24.295199 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:24.295208 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:24.295216 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:24.295416 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:24.295796 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:24.295801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:24.295804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:24.296513 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:24.296843 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:24.297223 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 22:59:24.297682 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1956,"thread_start_us":1710,"threads_started":1}
I20260809 22:59:24.299474 31635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } }
I20260809 22:59:24.299914 31635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:24.300040 31635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66318d8fd4694039bb253dc9c4f3640e, State: Initialized, Role: FOLLOWER
I20260809 22:59:24.300626 31635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } }
I20260809 22:59:24.300863 31635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:24.301004 31635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:24.301209 31635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:24.303689 31635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } }
I20260809 22:59:24.304260 31635 leader_election.cc:302] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66318d8fd4694039bb253dc9c4f3640e; no voters:
I20260809 22:59:24.304852 31635 leader_election.cc:288] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:24.304976 31641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:24.305692 31641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 1 LEADER]: Becoming Leader. State: Replica: 66318d8fd4694039bb253dc9c4f3640e, State: Running, Role: LEADER
I20260809 22:59:24.306208 31641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } }
I20260809 22:59:24.306918 31635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:24.309052 31643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66318d8fd4694039bb253dc9c4f3640e. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } }
I20260809 22:59:24.309270 31642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } }
I20260809 22:59:24.309552 31643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:24.309599 31642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:24.311373 31652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:24.315841 31652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:24.323930 31652 catalog_manager.cc:1347] Generated new cluster ID: 7c1eb2004dfb4568ab7d7d49fe0c9d62
I20260809 22:59:24.324100 31652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:24.353250 31652 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:24.354605 31652 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:24.364321 31652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: Generated new TSK 0
I20260809 22:59:24.365304 31652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab.
Entry for principal oryx/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab.
I20260809 22:59:24.433039 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46425
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab
--principal=oryx/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:24.775880 31665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:24.776226 31665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:24.776400 31665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:24.783643 31665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:24.783807 31665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:24.784000 31665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:24.793628 31665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab
--principal=oryx/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:24.794999 31665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:24.796314 31665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:24.809880 31671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:24.809902 31673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:24.809947 31675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:24.810791 31665 server_base.cc:1034] running on GCE node
I20260809 22:59:24.811456 31665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:24.812530 31665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:24.813762 31665 hybrid_clock.cc:648] HybridClock initialized: now 1786316364813722 us; error 55 us; skew 500 ppm
I20260809 22:59:24.814108 31665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:24 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316364, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:24.817744 31665 init.cc:377] Logged in from keytab as oryx/127.24.31.193@KRBTEST.COM (short username oryx)
I20260809 22:59:24.819211 31665 webserver.cc:460] Webserver started at http://127.24.31.193:35811/ using document root <none> and password file <none>
I20260809 22:59:24.819875 31665 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:24.819989 31665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:24.820238 31665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:24.823446 31665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "68596e5ddb484442af4a90aaa1af1356"
format_stamp: "Formatted at 2026-08-09 22:59:24 on dist-test-slave-vxj2"
server_key: "94da196900b5e89ebe93fc43452483db"
server_key_iv: "b275d3173217dbbcac42701c973242eb"
server_key_version: "encryptionkey@0"
I20260809 22:59:24.824343 31665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "68596e5ddb484442af4a90aaa1af1356"
format_stamp: "Formatted at 2026-08-09 22:59:24 on dist-test-slave-vxj2"
server_key: "94da196900b5e89ebe93fc43452483db"
server_key_iv: "b275d3173217dbbcac42701c973242eb"
server_key_version: "encryptionkey@0"
I20260809 22:59:24.830250 31665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 22:59:24.833894 31683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:24.835278 31665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:59:24.835460 31665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "68596e5ddb484442af4a90aaa1af1356"
format_stamp: "Formatted at 2026-08-09 22:59:24 on dist-test-slave-vxj2"
server_key: "94da196900b5e89ebe93fc43452483db"
server_key_iv: "b275d3173217dbbcac42701c973242eb"
server_key_version: "encryptionkey@0"
I20260809 22:59:24.835672 31665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:24.860301 31665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:24.865306 31665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:24.865614 31665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:24.866645 31665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:24.868023 31665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:24.868098 31665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:24.868249 31665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:24.868330 31665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:24.902143 31665 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:36033
I20260809 22:59:24.902197 31810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:36033 every 8 connection(s)
I20260809 22:59:24.903486 31665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:24.910379 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 31665
I20260809 22:59:24.910660 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:59:24.911343 24703 external_mini_cluster.cc:1428] Setting key bef033432a9fc2b494b9d6696f0ea9f1
Aug 09 22:59:24 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316364, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:24.925698 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:24.906182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:45319 (local address 127.24.31.254:46425)
0809 22:59:24.906686 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:24.906691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:24.907751 (+ 1060us) server_negotiation.cc:407] Connection header received
0809 22:59:24.909738 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:24.909744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:24.909899 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:24.910024 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:24.911931 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:24.912829 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:24.914590 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:24.915414 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:24.915562 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:24.919074 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:24.919082 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:24.919089 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:24.919135 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:24.922179 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:24.923189 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:24.923194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:24.923198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:24.923345 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:24.923869 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:24.923872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:24.923874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:24.924152 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:24.924295 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:24.925241 (+ 946us) server_negotiation.cc:299] Negotiation successful
0809 22:59:24.925443 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":302,"threads_started":1}
I20260809 22:59:24.927284 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:24.906730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.193:45319)
0809 22:59:24.907572 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:24.907630 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:24.909459 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:24.910332 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:24.910340 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:24.911018 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:24.911732 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:24.911751 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:24.913016 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:24.913027 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:24.914421 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:24.914434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:24.915628 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:24.915633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:24.915834 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:24.916548 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:24.916571 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:24.918842 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:24.922332 (+ 3490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:24.922344 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:24.922364 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:24.923040 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:24.923459 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:24.923467 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:24.923473 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:24.923756 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:24.924279 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:24.924290 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:24.924764 (+ 474us) client_negotiation.cc:769] Sending connection context
0809 22:59:24.925087 (+ 323us) client_negotiation.cc:240] Negotiation successful
0809 22:59:24.925744 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":274,"threads_started":1}
I20260809 22:59:24.928951 31811 heartbeater.cc:347] Connected to a master server at 127.24.31.254:46425
I20260809 22:59:24.929353 31811 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:24.930159 31811 heartbeater.cc:511] Master 127.24.31.254:46425 requested a full tablet report, sending...
I20260809 22:59:24.932504 31598 ts_manager.cc:194] Registered new tserver with Master: 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033)
I20260809 22:59:24.934038 31598 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:45319
WARNING: no policy specified for oryx/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab.
Entry for principal oryx/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab.
I20260809 22:59:24.966737 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46425
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab
--principal=oryx/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:25.298854 31819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:25.299144 31819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:25.299312 31819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:25.306854 31819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:25.306964 31819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:25.307199 31819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:25.317251 31819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab
--principal=oryx/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:25.318710 31819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:25.320076 31819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:25.333726 31825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:25.333725 31826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:25.333746 31829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:25.334969 31819 server_base.cc:1034] running on GCE node
I20260809 22:59:25.335636 31819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:25.336828 31819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:25.338124 31819 hybrid_clock.cc:648] HybridClock initialized: now 1786316365338060 us; error 51 us; skew 500 ppm
I20260809 22:59:25.338476 31819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:25 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316365, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:25.346186 31819 init.cc:377] Logged in from keytab as oryx/127.24.31.194@KRBTEST.COM (short username oryx)
I20260809 22:59:25.347762 31819 webserver.cc:460] Webserver started at http://127.24.31.194:46673/ using document root <none> and password file <none>
I20260809 22:59:25.348417 31819 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:25.348520 31819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:25.348850 31819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:25.351840 31819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "a6df5178167940189db0b8836ca61bbe"
format_stamp: "Formatted at 2026-08-09 22:59:25 on dist-test-slave-vxj2"
server_key: "ca16e865f6074eb2f2d60cff5ef632f6"
server_key_iv: "1dd1f3540fbccd9ce210f69f9fbd068b"
server_key_version: "encryptionkey@0"
I20260809 22:59:25.352659 31819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "a6df5178167940189db0b8836ca61bbe"
format_stamp: "Formatted at 2026-08-09 22:59:25 on dist-test-slave-vxj2"
server_key: "ca16e865f6074eb2f2d60cff5ef632f6"
server_key_iv: "1dd1f3540fbccd9ce210f69f9fbd068b"
server_key_version: "encryptionkey@0"
I20260809 22:59:25.358317 31819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 22:59:25.361775 31836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:25.363068 31819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:59:25.363247 31819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "a6df5178167940189db0b8836ca61bbe"
format_stamp: "Formatted at 2026-08-09 22:59:25 on dist-test-slave-vxj2"
server_key: "ca16e865f6074eb2f2d60cff5ef632f6"
server_key_iv: "1dd1f3540fbccd9ce210f69f9fbd068b"
server_key_version: "encryptionkey@0"
I20260809 22:59:25.363507 31819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:25.387884 31819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:25.393486 31819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:25.393790 31819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:25.394876 31819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:25.396198 31819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:25.396273 31819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:25.396366 31819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:25.396417 31819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:25.430713 31819 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:45727
I20260809 22:59:25.430820 31966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:45727 every 8 connection(s)
I20260809 22:59:25.432014 31819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:25.433960 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 31819
I20260809 22:59:25.434166 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:59:25.434672 24703 external_mini_cluster.cc:1428] Setting key e03cc24fdc2d6498d8fc26d574dc18dc
Aug 09 22:59:25 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316365, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:25.453274 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:25.434893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:34221 (local address 127.24.31.254:46425)
0809 22:59:25.435396 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:25.435400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:25.436644 (+ 1244us) server_negotiation.cc:407] Connection header received
0809 22:59:25.438487 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:25.438492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:25.438588 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:25.438703 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:25.440372 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:25.441221 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:25.442744 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:25.443160 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:25.443290 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:25.447594 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:25.447606 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:25.447609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:25.447641 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:25.450311 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:25.451016 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:25.451023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:25.451026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:25.451152 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:25.451566 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:25.451572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:25.451575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:25.451847 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:25.452016 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:25.452661 (+ 645us) server_negotiation.cc:299] Negotiation successful
0809 22:59:25.452878 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":242,"threads_started":1}
I20260809 22:59:25.454185 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:25.435590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.194:34221)
0809 22:59:25.436428 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:25.436487 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:25.438225 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:25.438979 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:25.438988 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:25.439513 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:25.440202 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:25.440223 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:25.441357 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:25.441363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:25.442584 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:25.442597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:25.443402 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:25.443409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:25.443645 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:25.444376 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:25.444411 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:25.447353 (+ 2942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:25.450453 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:25.450464 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:25.450478 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:25.450887 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:25.451262 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:25.451266 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:25.451270 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:25.451455 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:25.452001 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:25.452008 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:25.452311 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 22:59:25.452568 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 22:59:25.453072 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":353,"threads_started":1}
I20260809 22:59:25.455588 31967 heartbeater.cc:347] Connected to a master server at 127.24.31.254:46425
I20260809 22:59:25.455976 31967 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:25.456969 31967 heartbeater.cc:511] Master 127.24.31.254:46425 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.24.31.195@KRBTEST.COM; defaulting to no policy
I20260809 22:59:25.458838 31598 ts_manager.cc:194] Registered new tserver with Master: a6df5178167940189db0b8836ca61bbe (127.24.31.194:45727)
I20260809 22:59:25.459656 31598 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:34221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab.
Entry for principal oryx/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab.
I20260809 22:59:25.488477 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46425
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab
--principal=oryx/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:25.815292 31974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:25.815562 31974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:25.815743 31974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:25.823087 31974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:25.823215 31974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:25.823395 31974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:25.833062 31974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab
--principal=oryx/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:25.834401 31974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:25.835682 31974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:25.849112 31981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:25.849200 31983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:25.849211 31980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:25.850288 31974 server_base.cc:1034] running on GCE node
I20260809 22:59:25.850800 31974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:25.851662 31974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:25.852872 31974 hybrid_clock.cc:648] HybridClock initialized: now 1786316365852837 us; error 27 us; skew 500 ppm
I20260809 22:59:25.853207 31974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:25 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316365, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:25.856231 31974 init.cc:377] Logged in from keytab as oryx/127.24.31.195@KRBTEST.COM (short username oryx)
I20260809 22:59:25.857686 31974 webserver.cc:460] Webserver started at http://127.24.31.195:43811/ using document root <none> and password file <none>
I20260809 22:59:25.858309 31974 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:25.858407 31974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:25.858644 31974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:25.861636 31974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "d25cc28128cc4c5093710787c012491a"
format_stamp: "Formatted at 2026-08-09 22:59:25 on dist-test-slave-vxj2"
server_key: "ed4769e57b2437a88af59e98f67877c2"
server_key_iv: "522436599e5593395a1d7237a1e9fdd4"
server_key_version: "encryptionkey@0"
I20260809 22:59:25.862423 31974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "d25cc28128cc4c5093710787c012491a"
format_stamp: "Formatted at 2026-08-09 22:59:25 on dist-test-slave-vxj2"
server_key: "ed4769e57b2437a88af59e98f67877c2"
server_key_iv: "522436599e5593395a1d7237a1e9fdd4"
server_key_version: "encryptionkey@0"
I20260809 22:59:25.867584 31974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:59:25.870867 31990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:25.872107 31974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:59:25.872270 31974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "d25cc28128cc4c5093710787c012491a"
format_stamp: "Formatted at 2026-08-09 22:59:25 on dist-test-slave-vxj2"
server_key: "ed4769e57b2437a88af59e98f67877c2"
server_key_iv: "522436599e5593395a1d7237a1e9fdd4"
server_key_version: "encryptionkey@0"
I20260809 22:59:25.872473 31974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:25.884568 31974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:25.888226 31974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:25.888505 31974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:25.889420 31974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:25.890667 31974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:25.890741 31974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:25.890904 31974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:25.890956 31974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:25.925336 31974 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:43257
I20260809 22:59:25.925354 32122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:43257 every 8 connection(s)
I20260809 22:59:25.926789 31974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:25.929451 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 31974
I20260809 22:59:25.929639 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:59:25.930173 24703 external_mini_cluster.cc:1428] Setting key c76d43cf510e1d82a0dfb4b2dc525de8
I20260809 22:59:25.937124 31811 heartbeater.cc:503] Master 127.24.31.254:46425 was elected leader, sending a full tablet report...
Aug 09 22:59:25 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316365, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:25.948068 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:25.929479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:49985 (local address 127.24.31.254:46425)
0809 22:59:25.929735 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:25.929742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:25.931278 (+ 1536us) server_negotiation.cc:407] Connection header received
0809 22:59:25.933347 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:25.933353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:25.933446 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:25.933569 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:25.935013 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:25.935747 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:25.937373 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:25.937883 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:25.938018 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:25.941691 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:25.941695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:25.941698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:25.941732 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:25.944825 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:25.945748 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:25.945753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:25.945757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:25.945886 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:25.946322 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:25.946327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:25.946331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:25.946632 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:25.946809 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:25.947501 (+ 692us) server_negotiation.cc:299] Negotiation successful
0809 22:59:25.947734 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260809 22:59:25.949070 32125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:25.930013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.195:49985)
0809 22:59:25.930859 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:25.930896 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:25.933025 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:25.933793 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:25.933801 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:25.934294 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:25.934846 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:25.934863 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:25.935900 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:25.935905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:25.937212 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:25.937229 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:25.938089 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:25.938093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:25.938297 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:25.939063 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:25.939085 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:25.941506 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:25.944972 (+ 3466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:25.944982 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:25.944997 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:25.945623 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:25.945996 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:25.946001 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:25.946005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:25.946202 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:25.946768 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:25.946775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:25.947123 (+ 348us) client_negotiation.cc:769] Sending connection context
0809 22:59:25.947388 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 22:59:25.947919 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":356,"threads_started":1}
I20260809 22:59:25.950423 32123 heartbeater.cc:347] Connected to a master server at 127.24.31.254:46425
I20260809 22:59:25.950735 32123 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:25.951459 32123 heartbeater.cc:511] Master 127.24.31.254:46425 requested a full tablet report, sending...
I20260809 22:59:25.952867 31598 ts_manager.cc:194] Registered new tserver with Master: d25cc28128cc4c5093710787c012491a (127.24.31.195:43257)
I20260809 22:59:25.953687 31598 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:49985
I20260809 22:59:25.957844 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.31.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.253.keytab.
Entry for principal oryx/127.24.31.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.253.keytab.
I20260809 22:59:26.010891 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.253:38021
--webserver_interface=127.24.31.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.253.keytab
--principal=oryx/127.24.31.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:46425,127.24.31.253:38021
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:26.344296 32128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:26.344578 32128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:26.344758 32128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:26.352029 32128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:26.352138 32128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:26.352267 32128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:26.352349 32128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:26.352428 32128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:26.362314 32128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:46425,127.24.31.253:38021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.253.keytab
--principal=oryx/127.24.31.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.253:38021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.31.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
W20260809 22:59:26.362696 32128 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.31.254:46425,127.24.31.253:38021'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 22:59:26.363813 32128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:26.365120 32128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:26.376914 32133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:26.376950 32134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:26.377472 32136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:26.378360 32128 server_base.cc:1034] running on GCE node
I20260809 22:59:26.379134 32128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:26.380648 32128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:26.381894 32128 hybrid_clock.cc:648] HybridClock initialized: now 1786316366381829 us; error 43 us; skew 500 ppm
I20260809 22:59:26.382251 32128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:26 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316366, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:26.385962 32128 init.cc:377] Logged in from keytab as oryx/127.24.31.253@KRBTEST.COM (short username oryx)
I20260809 22:59:26.387454 32128 webserver.cc:460] Webserver started at http://127.24.31.253:43555/ using document root <none> and password file <none>
I20260809 22:59:26.388190 32128 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:26.388335 32128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:26.388680 32128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:26.391855 32128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data/instance:
uuid: "93cb005232ed48a78e2d5b8898761e45"
format_stamp: "Formatted at 2026-08-09 22:59:26 on dist-test-slave-vxj2"
server_key: "84401e1f8c5b384c2e40c6947b4dba68"
server_key_iv: "53d99619b8a1aac5c9bcb5643071bbf3"
server_key_version: "encryptionkey@0"
I20260809 22:59:26.392781 32128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal/instance:
uuid: "93cb005232ed48a78e2d5b8898761e45"
format_stamp: "Formatted at 2026-08-09 22:59:26 on dist-test-slave-vxj2"
server_key: "84401e1f8c5b384c2e40c6947b4dba68"
server_key_iv: "53d99619b8a1aac5c9bcb5643071bbf3"
server_key_version: "encryptionkey@0"
I20260809 22:59:26.398711 32128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 22:59:26.402278 32143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:26.403590 32128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:59:26.403771 32128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal
uuid: "93cb005232ed48a78e2d5b8898761e45"
format_stamp: "Formatted at 2026-08-09 22:59:26 on dist-test-slave-vxj2"
server_key: "84401e1f8c5b384c2e40c6947b4dba68"
server_key_iv: "53d99619b8a1aac5c9bcb5643071bbf3"
server_key_version: "encryptionkey@0"
I20260809 22:59:26.404014 32128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:26.427438 32128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:26.431631 32128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:26.432003 32128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:26.451109 32128 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.253:38021
I20260809 22:59:26.451128 32207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.253:38021 every 8 connection(s)
I20260809 22:59:26.452505 32128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
I20260809 22:59:26.454356 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 32128
I20260809 22:59:26.456311 32208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:26.462458 31967 heartbeater.cc:503] Master 127.24.31.254:46425 was elected leader, sending a full tablet report...
I20260809 22:59:26.464195 32208 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:59:26 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.253@KRBTEST.COM
Aug 09 22:59:26 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316366, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.253@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:26.479838 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.456488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59868 (local address 127.24.31.253:38021)
0809 22:59:26.457293 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:26.457305 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:26.457352 (+ 47us) server_negotiation.cc:407] Connection header received
0809 22:59:26.459019 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:26.459040 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:26.459404 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:26.459798 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:26.461338 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.462868 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.464683 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.465447 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.465574 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.471562 (+ 5988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:26.471570 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:26.471579 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:26.471618 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:26.475098 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.475987 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.475998 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.476007 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.476256 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.476741 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.476746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.476749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.477409 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:26.477644 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:26.478259 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 22:59:26.478651 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":327,"threads_started":1}
I20260809 22:59:26.485599 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.467083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:53788)
0809 22:59:26.469654 (+ 2571us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:26.469711 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:26.469923 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:26.470332 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:26.470341 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:26.471198 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:26.471561 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.471572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.472998 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.473001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.474380 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.474391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.475459 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.475465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.475650 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.476343 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:26.476378 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:26.479076 (+ 2698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:26.483263 (+ 4187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.483270 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.483275 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.483695 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.484031 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.484033 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.484036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.484157 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.484682 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:26.484687 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:26.484825 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 22:59:26.485043 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 22:59:26.485328 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2350,"thread_start_us":2245,"threads_started":1}
I20260809 22:59:26.485637 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.467031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.24.31.254:46425)
0809 22:59:26.467544 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:26.467548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:26.469822 (+ 2274us) server_negotiation.cc:407] Connection header received
0809 22:59:26.470084 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:26.470088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:26.470155 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:26.470239 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:26.471706 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.472835 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.474534 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.475261 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.475362 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.479275 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:26.479281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:26.479285 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:26.479325 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:26.483148 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.483817 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.483821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.483823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.483941 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.484266 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.484269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.484271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.484580 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:26.484692 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:26.485198 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 22:59:26.485383 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":266,"threads_started":1}
I20260809 22:59:26.487498 32208 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:59:26 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316366, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.253@KRBTEST.COM for oryx/127.24.31.253@KRBTEST.COM
I20260809 22:59:26.499558 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.488349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.0.0.1:59884)
0809 22:59:26.488540 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:26.488559 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:26.488681 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:26.488948 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:26.488952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:26.489373 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:26.489504 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.489512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.490460 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.490465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.491321 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.491332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.491978 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.491983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.492176 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.492735 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:26.492751 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:26.494781 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:26.497407 (+ 2626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.497411 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.497413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.497779 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.498083 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.498086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.498088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.498208 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.498882 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:26.498885 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:26.499013 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 22:59:26.499120 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 22:59:26.499313 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":3}
I20260809 22:59:26.499929 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.488335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59884 (local address 127.24.31.253:38021)
0809 22:59:26.488526 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:26.488532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:26.488625 (+ 93us) server_negotiation.cc:407] Connection header received
0809 22:59:26.488792 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:26.488795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:26.488844 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:26.488933 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:26.489627 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.490304 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.491458 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.491844 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.491968 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.494953 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:26.494957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:26.494960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:26.494987 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:26.497290 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.497874 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.497878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.497880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.497992 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.498310 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.498314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.498317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.498771 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:26.498902 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:26.499219 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 22:59:26.499440 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":58}
I20260809 22:59:26.504894 32208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: Bootstrap starting.
I20260809 22:59:26.509313 32208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:26.510459 32208 log.cc:824] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:26.513406 32208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: No bootstrap required, opened a new log
I20260809 22:59:26.520768 32208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } }
I20260809 22:59:26.521226 32208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:26.521392 32208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93cb005232ed48a78e2d5b8898761e45, State: Initialized, Role: FOLLOWER
I20260809 22:59:26.522070 32208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } }
I20260809 22:59:26.523003 32215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } } }
I20260809 22:59:26.523396 32215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:59:26.524313 32208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:59:26.533687 32228 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:59:26.533829 32228 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:59:26.536827 32128 master_runner.cc:428] Detected that this master 93cb005232ed48a78e2d5b8898761e45 is joining an existing cluster
I20260809 22:59:26.536931 32128 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.31.253:38021
WARNING: no policy specified for oryx/127.24.31.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.252@KRBTEST.COM" created.
I20260809 22:59:26.577988 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.548548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.24.31.253:38021)
0809 22:59:26.548856 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:26.548862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:26.560283 (+ 11421us) server_negotiation.cc:407] Connection header received
0809 22:59:26.560443 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:26.560454 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:26.560520 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:26.560613 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:26.562668 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.563845 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.566297 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.567314 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.567429 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.570246 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:26.570253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:26.570266 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:26.570304 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:26.574850 (+ 4546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.575695 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.575701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.575705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.575859 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.576337 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.576344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.576349 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.576699 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:26.576843 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:26.577394 (+ 551us) server_negotiation.cc:299] Negotiation successful
0809 22:59:26.577612 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 22:59:26.578558 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.548965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.24.31.254:46425)
0809 22:59:26.549198 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:26.549206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:26.559825 (+ 10619us) server_negotiation.cc:407] Connection header received
0809 22:59:26.560224 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:26.560231 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:26.560352 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:26.560479 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:26.562227 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.563368 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.565576 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.566321 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.566444 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.569397 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:26.569404 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:26.569409 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:26.569451 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:26.574220 (+ 4769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.575152 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.575161 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.575167 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.575409 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.575994 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.576002 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.576009 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.576499 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:26.576645 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:26.577531 (+ 886us) server_negotiation.cc:299] Negotiation successful
0809 22:59:26.577795 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:59:26.579042 32235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.552979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.0.0.1:59898)
0809 22:59:26.560172 (+ 7193us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:26.560198 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:26.560333 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:26.560799 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:26.560808 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:26.561567 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:26.562506 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.562519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.564091 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.564098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.566063 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.566112 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.567788 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.567798 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.568029 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.568706 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:26.568735 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:26.570029 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:26.575004 (+ 4975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.575009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.575013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.575563 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.575994 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.575999 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.576004 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.576187 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.576911 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:26.576916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:26.577069 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 22:59:26.578275 (+ 1206us) client_negotiation.cc:240] Negotiation successful
0809 22:59:26.578566 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7027,"mutex_wait_us":26,"thread_start_us":6250,"threads_started":1}
I20260809 22:59:26.580103 32236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.548559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:53802)
0809 22:59:26.559611 (+ 11052us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:26.559645 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:26.560016 (+ 371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:26.560514 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:26.560518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:26.561548 (+ 1030us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:26.562025 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.562038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.563562 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.563567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.565410 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.565425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.566802 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.566808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.567002 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.567821 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:26.567845 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:26.569130 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:26.574368 (+ 5238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.574376 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.574382 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.575002 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.575575 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.575583 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.575589 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.575766 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.576735 (+ 969us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:26.576742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:26.576978 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 22:59:26.579014 (+ 2036us) client_negotiation.cc:240] Negotiation successful
0809 22:59:26.579658 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10884,"mutex_wait_us":30,"spinlock_wait_cycles":28032,"thread_start_us":11240,"threads_started":1}
Aug 09 22:59:26 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316364, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.254@KRBTEST.COM for oryx/127.24.31.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.252.keytab.
Entry for principal oryx/127.24.31.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.252.keytab.
I20260809 22:59:26.600447 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.252:33965
--webserver_interface=127.24.31.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.252.keytab
--principal=oryx/127.24.31.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:46425,127.24.31.253:38021,127.24.31.252:33965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:59:26.604839 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.587493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59910 (local address 127.24.31.253:38021)
0809 22:59:26.587656 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:26.587662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:26.588011 (+ 349us) server_negotiation.cc:407] Connection header received
0809 22:59:26.588445 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:26.588451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:26.588555 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:26.588661 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:26.590172 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.591369 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.592730 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.593488 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.593629 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.597581 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:26.597588 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:26.597593 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:26.597630 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:26.601400 (+ 3770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.602408 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.602415 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.602418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.602583 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.603058 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.603063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.603067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.603452 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:26.603601 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:26.604189 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 22:59:26.604458 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 22:59:26.605336 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.587111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.0.0.1:59910)
0809 22:59:26.587872 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:26.587924 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:26.588171 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:26.588741 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:26.588751 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:26.589710 (+ 959us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:26.590011 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.590022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.591509 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.591513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.592577 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.592589 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.593655 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.593660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.593789 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.594783 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:26.594804 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:26.597402 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:26.601559 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.601578 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.601590 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.602270 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.602710 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.602717 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.602724 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.602936 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.603601 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:26.603613 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:26.603849 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 22:59:26.604078 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 22:59:26.604500 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":551,"threads_started":1}
I20260809 22:59:26.607856 31598 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.31.253:38021: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "93cb005232ed48a78e2d5b8898761e45"
member_type: NON_VOTER
last_known_addr {
host: "127.24.31.253"
port: 38021
}
attrs {
promote: true
}
}
dest_uuid: "66318d8fd4694039bb253dc9c4f3640e"
cas_config_opid_index: -1
I20260809 22:59:26.609102 31598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } }
I20260809 22:59:26.613497 32173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:26.617478 32173 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Refusing update from remote peer 66318d8fd4694039bb253dc9c4f3640e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 22:59:26.618937 32242 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 93cb005232ed48a78e2d5b8898761e45 (127.24.31.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } }
I20260809 22:59:26.620653 32243 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Connected to new peer: Peer: permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:26.624976 32242 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.31.253:38021
I20260809 22:59:26.626466 32128 master.cc:537] Master@127.24.31.253:38021 shutting down...
I20260809 22:59:26.636587 32242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } }
I20260809 22:59:26.636881 32242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:26.643797 32128 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:26.644341 32128 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:26.644583 32128 tablet_replica.cc:330] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: stopping tablet replica
W20260809 22:59:26.646620 31585 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 22:59:26.655256 31585 connection.cc:535] client connection to 127.24.31.253:38021 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 22:59:26.655500 31585 consensus_peers.cc:487] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e -> Peer 93cb005232ed48a78e2d5b8898761e45 (127.24.31.253:38021): Couldn't send request to peer 93cb005232ed48a78e2d5b8898761e45. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260809 22:59:26.659556 32128 master.cc:552] Master@127.24.31.253:38021 shutdown complete.
I20260809 22:59:26.659703 32128 master_runner.cc:305] Clearing existing system tablet
I20260809 22:59:26.660190 32128 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:26.660291 32128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:26.663475 32128 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 22:59:26.665213 32251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:26.665608 32128 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:26.665886 32128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 22:59:26.665983 32128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal
uuid: "93cb005232ed48a78e2d5b8898761e45"
format_stamp: "Formatted at 2026-08-09 22:59:26 on dist-test-slave-vxj2"
server_key: "84401e1f8c5b384c2e40c6947b4dba68"
server_key_iv: "53d99619b8a1aac5c9bcb5643071bbf3"
server_key_version: "encryptionkey@0"
I20260809 22:59:26.670835 32128 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:26.678959 32128 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 22:59:26.679215 32128 log.cc:1196] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 22:59:26.679806 32128 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: Deleting consensus metadata
I20260809 22:59:26.680169 32128 master_runner.cc:315] Copying system tablet from 127.24.31.254:46425
I20260809 22:59:26.683485 32128 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: tablet copy: Beginning tablet copy session from remote peer at address 127.24.31.254:46425
I20260809 22:59:26.700968 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.684901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.24.31.254:46425)
0809 22:59:26.685211 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:26.685220 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:26.685456 (+ 236us) server_negotiation.cc:407] Connection header received
0809 22:59:26.685824 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:26.685833 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:26.685932 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:26.686060 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:26.687229 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.688483 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.690475 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.691166 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.691313 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.693798 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:26.693806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:26.693811 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:26.693858 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:26.697726 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.698580 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.698585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.698589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.698768 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:26.699239 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:26.699244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:26.699248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:26.699665 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:26.699839 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:26.700334 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 22:59:26.700577 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 22:59:26.701411 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:26.684393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:53806)
0809 22:59:26.685314 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:26.685344 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:26.685592 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:26.686109 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:26.686114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:26.686712 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:26.687039 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.687054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.688669 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.688675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.690272 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:26.690288 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:26.691360 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:26.691368 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:26.691570 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:26.692365 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:26.692395 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:26.693544 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:26.697872 (+ 4328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.697878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.697884 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.698426 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.698900 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:26.698905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:26.698909 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:26.699102 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:26.699809 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:26.699814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:26.700005 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 22:59:26.700168 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 22:59:26.700547 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":828,"threads_started":1}
I20260809 22:59:26.702569 31620 tablet_copy_service.cc:140] P 66318d8fd4694039bb253dc9c4f3640e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 93cb005232ed48a78e2d5b8898761e45 ({username='oryx', principal='oryx/127.24.31.253@KRBTEST.COM'} at 127.0.0.1:53806)
I20260809 22:59:26.702878 31620 tablet_copy_service.cc:161] P 66318d8fd4694039bb253dc9c4f3640e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 93cb005232ed48a78e2d5b8898761e45 at {username='oryx', principal='oryx/127.24.31.253@KRBTEST.COM'} at 127.0.0.1:53806: session id = 93cb005232ed48a78e2d5b8898761e45-00000000000000000000000000000000
I20260809 22:59:26.707656 31620 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 22:59:26.710062 32128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 22:59:26.718237 32128 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: tablet copy: Starting download of 0 data blocks...
I20260809 22:59:26.718631 32128 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: tablet copy: Starting download of 1 WAL segments...
I20260809 22:59:26.721894 32128 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 22:59:26.726994 31620 tablet_copy_service.cc:342] P 66318d8fd4694039bb253dc9c4f3640e: Request end of tablet copy session 93cb005232ed48a78e2d5b8898761e45-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.31.253@KRBTEST.COM'} at 127.0.0.1:53806
I20260809 22:59:26.727269 31620 tablet_copy_service.cc:433] P 66318d8fd4694039bb253dc9c4f3640e: ending tablet copy session 93cb005232ed48a78e2d5b8898761e45-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 93cb005232ed48a78e2d5b8898761e45
I20260809 22:59:26.740478 32128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:26.743521 32259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:26.746591 32260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:26.759490 32262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:26.760444 32128 server_base.cc:1034] running on GCE node
I20260809 22:59:26.760815 32128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:26.765404 32128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:26.766651 32128 hybrid_clock.cc:648] HybridClock initialized: now 1786316366766555 us; error 52 us; skew 500 ppm
I20260809 22:59:26.766906 32128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:26 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316366, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:26.769282 32128 init.cc:377] Logged in from keytab as oryx/127.24.31.253@KRBTEST.COM (short username oryx)
I20260809 22:59:26.772979 32128 webserver.cc:460] Webserver started at http://127.24.31.253:35635/ using document root <none> and password file <none>
I20260809 22:59:26.773527 32128 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:26.773653 32128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:26.777572 32128 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 22:59:26.779636 32271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:26.780095 32128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 22:59:26.780316 32128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal
uuid: "93cb005232ed48a78e2d5b8898761e45"
format_stamp: "Formatted at 2026-08-09 22:59:26 on dist-test-slave-vxj2"
server_key: "84401e1f8c5b384c2e40c6947b4dba68"
server_key_iv: "53d99619b8a1aac5c9bcb5643071bbf3"
server_key_version: "encryptionkey@0"
I20260809 22:59:26.780580 32128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:26.818145 32128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:26.823196 32128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:26.849385 32128 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.253:38021
I20260809 22:59:26.849406 32327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.253:38021 every 8 connection(s)
I20260809 22:59:26.851065 32128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
I20260809 22:59:26.856472 32329 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:59:26.860467 32329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: Bootstrap starting.
I20260809 22:59:26.878345 32329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:59:26.878907 32329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: Bootstrap complete.
I20260809 22:59:26.879987 32329 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } }
I20260809 22:59:26.880254 32329 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 93cb005232ed48a78e2d5b8898761e45, State: Initialized, Role: LEARNER
I20260809 22:59:26.880570 32329 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } }
I20260809 22:59:26.881520 32332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } }
I20260809 22:59:26.881920 32329 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:26.882021 32332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: This master's current role is: LEARNER
I20260809 22:59:26.895284 32347 catalog_manager.cc:1259] Loaded cluster ID: 7c1eb2004dfb4568ab7d7d49fe0c9d62
I20260809 22:59:26.895409 32347 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: loading cluster ID for follower catalog manager: success
I20260809 22:59:26.899053 32347 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: acquiring CA information for follower catalog manager: success
I20260809 22:59:26.901618 32347 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:59:26.956267 32123 heartbeater.cc:503] Master 127.24.31.254:46425 was elected leader, sending a full tablet report...
W20260809 22:59:27.038548 32239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:27.038880 32239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:27.039026 32239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:27.046309 32239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:27.046454 32239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:27.046567 32239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:27.046671 32239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:27.046743 32239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:27.056521 32239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:46425,127.24.31.253:38021,127.24.31.252:33965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.252.keytab
--principal=oryx/127.24.31.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.252:33965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.31.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:27.057874 32239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:27.059207 32239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:27.071300 32358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:27.071249 32356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:27.071343 32355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:27.077383 32239 server_base.cc:1034] running on GCE node
I20260809 22:59:27.078092 32239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:27.079430 32239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:27.080621 32239 hybrid_clock.cc:648] HybridClock initialized: now 1786316367080567 us; error 48 us; skew 500 ppm
I20260809 22:59:27.080992 32239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:27 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316367, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:27.084414 32239 init.cc:377] Logged in from keytab as oryx/127.24.31.252@KRBTEST.COM (short username oryx)
I20260809 22:59:27.086076 32239 webserver.cc:460] Webserver started at http://127.24.31.252:43515/ using document root <none> and password file <none>
I20260809 22:59:27.086859 32239 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:27.086963 32239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:27.087203 32239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:27.090049 32239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data/instance:
uuid: "bae2d3a95a0f4da695a51f5be350262f"
format_stamp: "Formatted at 2026-08-09 22:59:27 on dist-test-slave-vxj2"
server_key: "f24535cf0a7fc83bbe4a2d3cfbcbd638"
server_key_iv: "6e87e080f48a603e8d8b9003090af938"
server_key_version: "encryptionkey@0"
I20260809 22:59:27.090770 32239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal/instance:
uuid: "bae2d3a95a0f4da695a51f5be350262f"
format_stamp: "Formatted at 2026-08-09 22:59:27 on dist-test-slave-vxj2"
server_key: "f24535cf0a7fc83bbe4a2d3cfbcbd638"
server_key_iv: "6e87e080f48a603e8d8b9003090af938"
server_key_version: "encryptionkey@0"
I20260809 22:59:27.096179 32239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:59:27.100417 32367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:27.102059 32239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:27.102232 32239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal
uuid: "bae2d3a95a0f4da695a51f5be350262f"
format_stamp: "Formatted at 2026-08-09 22:59:27 on dist-test-slave-vxj2"
server_key: "f24535cf0a7fc83bbe4a2d3cfbcbd638"
server_key_iv: "6e87e080f48a603e8d8b9003090af938"
server_key_version: "encryptionkey@0"
I20260809 22:59:27.102483 32239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:27.102794 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.095504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.0.0.1:59926)
0809 22:59:27.095710 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.095730 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.095868 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.096719 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.096723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.096747 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:27.096908 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.096916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.098800 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.098804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.100502 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.100519 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.102059 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.102063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.102270 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.102312 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.102399 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.102555 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 22:59:27.102964 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.095693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59926 (local address 127.24.31.253:38021)
0809 22:59:27.096338 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.096344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.096365 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:59:27.096440 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.096445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.096549 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.096686 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:27.097065 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.098664 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.100646 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.101889 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.102043 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.102301 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.102448 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.102601 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":335,"threads_started":1}
I20260809 22:59:27.104172 32302 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 22:59:27.104714 32332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66318d8fd4694039bb253dc9c4f3640e. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } }
I20260809 22:59:27.104992 32332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: This master's current role is: LEARNER
I20260809 22:59:27.120604 32239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:27.124873 32239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:27.125175 32239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:27.144552 32239 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.252:33965
I20260809 22:59:27.144588 32424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.252:33965 every 8 connection(s)
I20260809 22:59:27.146006 32239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
I20260809 22:59:27.149785 32425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:27.155638 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 32239
I20260809 22:59:27.156504 32425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:59:27 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316367, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.252@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
Aug 09 22:59:27 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.252@KRBTEST.COM
I20260809 22:59:27.176558 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.159534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.24.31.254:46425)
0809 22:59:27.159691 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.159695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.160411 (+ 716us) server_negotiation.cc:407] Connection header received
0809 22:59:27.161477 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.161482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.161566 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.161678 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.162934 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.163660 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.165183 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.165576 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.165715 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.169701 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.169708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.169713 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.169754 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.172826 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.173923 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.173927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.173929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.174064 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.174592 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.174596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.174598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.175018 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.175185 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.176130 (+ 945us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.176315 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 22:59:27.178012 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.159559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:53814)
0809 22:59:27.160251 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.160285 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.161229 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.161736 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.161742 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.162391 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:27.162763 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.162780 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.163804 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.163810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.165017 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.165028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.165827 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.165834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.166105 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.166964 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:27.166995 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:27.169525 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:27.172989 (+ 3464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.173000 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.173012 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.173795 (+ 783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.174181 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.174192 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.174198 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.174432 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.175154 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:27.175160 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:27.175568 (+ 408us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.176006 (+ 438us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.176542 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":38,"thread_start_us":341,"threads_started":1}
I20260809 22:59:27.178373 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.158034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40908 (local address 127.24.31.252:33965)
0809 22:59:27.158836 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.158847 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.158909 (+ 62us) server_negotiation.cc:407] Connection header received
0809 22:59:27.161224 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.161245 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.161466 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.161659 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.163161 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.164321 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.165590 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.166191 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.166338 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.171165 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.171174 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.171185 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.171216 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.174897 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.175631 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.175640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.175646 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.175847 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.176302 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.176308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.176311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.176807 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.177134 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.177650 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.177883 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":577,"spinlock_wait_cycles":17792,"thread_start_us":362,"threads_started":1}
I20260809 22:59:27.179724 32425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:59:27 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316367, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.252@KRBTEST.COM for oryx/127.24.31.253@KRBTEST.COM
I20260809 22:59:27.193671 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.180643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.0.0.1:59942)
0809 22:59:27.180928 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.180947 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.181071 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.181629 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.181632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.182072 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:27.182273 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.182315 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.183477 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.183483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.184472 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.184484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.185227 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.185233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.185441 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.186200 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:27.186217 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:27.188216 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:27.191245 (+ 3029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.191251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.191255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.191635 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.192019 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.192025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.192028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.192168 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.192673 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:27.192679 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:27.192827 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.192989 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.193308 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260809 22:59:27.193671 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.180558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.24.31.253:38021)
0809 22:59:27.180839 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.180843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.181013 (+ 170us) server_negotiation.cc:407] Connection header received
0809 22:59:27.181278 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.181284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.181365 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.181466 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.182473 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.183330 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.184616 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.185066 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.185236 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.188401 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.188406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.188410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.188442 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.191079 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.191752 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.191759 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.191762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.191899 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.192272 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.192279 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.192282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.192557 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.192710 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.193081 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.193325 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260809 22:59:27.194923 32425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 22:59:27 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316367, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.252@KRBTEST.COM for oryx/127.24.31.252@KRBTEST.COM
I20260809 22:59:27.208108 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.195757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.24.31.252:33965)
0809 22:59:27.195939 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.195945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.195964 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:59:27.196106 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.196111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.196170 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.196289 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.196986 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.197791 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.199002 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.199425 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.199573 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.202760 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.202765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.202768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.202798 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.205291 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.205929 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.205936 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.205940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.206067 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.206464 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.206482 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.206486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.206814 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.206995 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.207394 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.207726 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":48}
I20260809 22:59:27.208129 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.195630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:33965 (local address 127.0.0.1:40918)
0809 22:59:27.195838 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.195859 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.195981 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.196302 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.196308 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.196694 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:27.196847 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.196856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.197920 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.197933 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.198849 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.198861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.199580 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.199586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.199847 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.200439 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:27.200456 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:27.202560 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:27.205415 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.205422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.205426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.205809 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.206178 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.206184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.206188 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.206349 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.206959 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:27.206964 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:27.207113 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.207280 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.207577 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":36}
I20260809 22:59:27.215309 32425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Bootstrap starting.
I20260809 22:59:27.218588 32425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:27.219625 32425 log.cc:824] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:27.222287 32425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: No bootstrap required, opened a new log
I20260809 22:59:27.229736 32425 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } }
I20260809 22:59:27.230072 32425 raft_consensus.cc:374] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:27.230195 32425 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae2d3a95a0f4da695a51f5be350262f, State: Initialized, Role: FOLLOWER
I20260809 22:59:27.230830 32425 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } }
I20260809 22:59:27.231750 32431 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } } }
I20260809 22:59:27.232247 32431 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:59:27.232990 32425 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: configured and running, proceeding with master startup.
W20260809 22:59:27.241889 32443 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 22:59:27.242058 32443 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 22:59:27.246367 32239 master_runner.cc:428] Detected that this master bae2d3a95a0f4da695a51f5be350262f is joining an existing cluster
I20260809 22:59:27.246515 32239 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.31.252:33965
I20260809 22:59:27.271387 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.251739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.24.31.254:46425)
0809 22:59:27.252190 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.252200 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.254530 (+ 2330us) server_negotiation.cc:407] Connection header received
0809 22:59:27.254754 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.254759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.254851 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.254960 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.256588 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.257844 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.260215 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.260901 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.261005 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.263774 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.263781 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.263786 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.263849 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.267745 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.268586 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.268592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.268595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.268763 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.269367 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.269372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.269375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.269787 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.269926 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.270743 (+ 817us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.270989 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285}
I20260809 22:59:27.272548 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.251131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:53816)
0809 22:59:27.254374 (+ 3243us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.254401 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.254593 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.255166 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.255171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.255953 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:27.256353 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.256368 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.258058 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.258069 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.260037 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.260055 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.261327 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.261333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.261538 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.262315 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:27.262340 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:27.263515 (+ 1175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:27.267906 (+ 4391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.267911 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.267914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.268431 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.268964 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.268970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.268975 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.269209 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.270051 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:27.270057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:27.270355 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.271617 (+ 1262us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.271994 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3056,"mutex_wait_us":1,"thread_start_us":5260,"threads_started":1}
I20260809 22:59:27.275225 32366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.251061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.24.31.253:38021)
0809 22:59:27.251268 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.251277 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.257225 (+ 5948us) server_negotiation.cc:407] Connection header received
0809 22:59:27.257473 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.257479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.257567 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.257634 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.259521 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.260581 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.262583 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.263233 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.263332 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.268174 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.268180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.268184 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.268225 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.271901 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.272752 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.272760 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.272767 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.272925 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.273510 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.273516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.273519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.273850 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.274009 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.274657 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.274900 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:59:27.276419 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.252620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.0.0.1:59956)
0809 22:59:27.257087 (+ 4467us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.257105 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.257326 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.257875 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.257882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.258630 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:27.259294 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.259365 (+ 71us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.260777 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.260784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.262420 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.262437 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.263601 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.263607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.263766 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.264391 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:27.264418 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:27.267959 (+ 3541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:27.272097 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.272101 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.272105 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.272596 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.273090 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.273095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.273098 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.273354 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.274104 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:27.274110 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:27.274329 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.275481 (+ 1152us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.275794 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4341,"thread_start_us":4248,"threads_started":1}
I20260809 22:59:27.284715 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 31665
I20260809 22:59:27.288046 32450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.251257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:33965 (local address 127.0.0.1:40928)
0809 22:59:27.256738 (+ 5481us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.256762 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.257124 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.257584 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.257588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.258474 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:27.258865 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.258881 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.260508 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.260513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.274262 (+ 13749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.274277 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.276948 (+ 2671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.276955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.277188 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.278747 (+ 1559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:27.278770 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:27.280051 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:27.284295 (+ 4244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.284300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.284304 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.284776 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.285324 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.285330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.285335 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.285493 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.286915 (+ 1422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:27.286921 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:27.287113 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.287266 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.287586 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5326,"thread_start_us":2744,"threads_started":1}
Aug 09 22:59:27 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316364, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.254@KRBTEST.COM for oryx/127.24.31.252@KRBTEST.COM
I20260809 22:59:27.289891 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.251958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40928 (local address 127.24.31.252:33965)
0809 22:59:27.252354 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.252362 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.256862 (+ 4500us) server_negotiation.cc:407] Connection header received
0809 22:59:27.257262 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.257267 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.257336 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.257399 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.259000 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.260313 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.275096 (+ 14783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.275875 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.275988 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.280258 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.280265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.280270 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.280308 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.284155 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.285009 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.285014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.285018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.285203 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.286326 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.286332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.286335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.286797 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.286962 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.289304 (+ 2342us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.289518 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":262}
I20260809 22:59:27.296582 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.278747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:33965 (local address 127.0.0.1:40944)
0809 22:59:27.279113 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.279134 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.279245 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.281081 (+ 1836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.281089 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.281548 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:27.281793 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.281804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.283705 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.283710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.285023 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.285041 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.286108 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.286114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.286326 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.286915 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:27.286936 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:27.289805 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:27.293909 (+ 4104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.293915 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.293919 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.294407 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.294828 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.294833 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.294837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.294981 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.295597 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:27.295603 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:27.295771 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.295883 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.296177 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260809 22:59:27.297677 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.279442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40944 (local address 127.24.31.252:33965)
0809 22:59:27.280672 (+ 1230us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.280676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.280695 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:59:27.280766 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.280769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.280832 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.281915 (+ 1083us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.282330 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.283541 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.285258 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.285965 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.286094 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.290040 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.290047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.290052 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.290089 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.293768 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.294572 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.294577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.294581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.294714 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.295126 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.295130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.295134 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.295470 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.296914 (+ 1444us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.297036 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.297254 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1113,"thread_start_us":967,"threads_started":1}
I20260809 22:59:27.299084 31598 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.31.252:33965: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f"
member_type: NON_VOTER
last_known_addr {
host: "127.24.31.252"
port: 33965
}
attrs {
promote: true
}
}
dest_uuid: "66318d8fd4694039bb253dc9c4f3640e"
cas_config_opid_index: 5
I20260809 22:59:27.300165 31598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } }
I20260809 22:59:27.302176 32243 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bae2d3a95a0f4da695a51f5be350262f (127.24.31.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.305992 32302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 LEARNER]: Refusing update from remote peer 66318d8fd4694039bb253dc9c4f3640e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 22:59:27.307626 32394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 0 FOLLOWER]: Advancing to term 1
Aug 09 22:59:27 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.193@KRBTEST.COM
I20260809 22:59:27.312541 32394 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 FOLLOWER]: Refusing update from remote peer 66318d8fd4694039bb253dc9c4f3640e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 22:59:27.324818 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.286295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.24.31.193:36033)
0809 22:59:27.298082 (+ 11787us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.298093 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.298126 (+ 33us) server_negotiation.cc:407] Connection header received
0809 22:59:27.298205 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.298211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.298427 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.300110 (+ 1683us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.300302 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.301660 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.304212 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.305460 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.313166 (+ 7706us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.313564 (+ 398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.313581 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.313608 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.313661 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.318966 (+ 5305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.320199 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.320212 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.320218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.320441 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.321078 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.321086 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.321092 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.321696 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.323627 (+ 1931us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.323848 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.324213 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11571,"thread_start_us":11627,"threads_started":1}
I20260809 22:59:27.326700 31805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:27.330452 32457 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Connected to new peer: Peer: permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 22:59:27.333107 32456 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:27.342482 32394 raft_consensus.cc:858] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } }
W20260809 22:59:27.342700 32394 raft_consensus.cc:1565] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } }
I20260809 22:59:27.346283 32243 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.31.252:33965
I20260809 22:59:27.347060 32239 master.cc:537] Master@127.24.31.252:33965 shutting down...
I20260809 22:59:27.357909 32302 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER bae2d3a95a0f4da695a51f5be350262f (127.24.31.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.361440 32456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.361774 32456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:27.363646 32332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.363950 32332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: This master's current role is: LEARNER
I20260809 22:59:27.365242 32457 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: attempting to promote NON_VOTER 93cb005232ed48a78e2d5b8898761e45 to VOTER
I20260809 22:59:27.366210 32457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } }
I20260809 22:59:27.376022 32302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 LEARNER]: Refusing update from remote peer 66318d8fd4694039bb253dc9c4f3640e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 22:59:27.376714 32243 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Connected to new peer: Peer: permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 22:59:27.378271 31583 consensus_peers.cc:487] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e -> Peer bae2d3a95a0f4da695a51f5be350262f (127.24.31.252:33965): Couldn't send request to peer bae2d3a95a0f4da695a51f5be350262f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 22:59:27.391785 32301 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 93cb005232ed48a78e2d5b8898761e45 (127.24.31.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.396119 32332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.396380 32332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:59:27.400831 32455 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 93cb005232ed48a78e2d5b8898761e45 (127.24.31.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.404531 32394 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 93cb005232ed48a78e2d5b8898761e45 (127.24.31.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } }
I20260809 22:59:27.409689 32243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.410066 32243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:27.410542 32455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.410938 32455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:27.411852 32469 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } }
I20260809 22:59:27.412295 32469 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 22:59:27.412207 32465 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } }
I20260809 22:59:27.412586 32465 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 22:59:27.412686 32431 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66318d8fd4694039bb253dc9c4f3640e. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: NON_VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: true } } }
I20260809 22:59:27.412882 32431 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 22:59:27.415900 32239 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 22:59:27.416399 32239 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 22:59:27.416582 32239 tablet_replica.cc:330] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: stopping tablet replica
I20260809 22:59:27.424727 32239 master.cc:552] Master@127.24.31.252:33965 shutdown complete.
I20260809 22:59:27.424811 32239 master_runner.cc:305] Clearing existing system tablet
I20260809 22:59:27.425231 32239 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:27.425333 32239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:27.433382 32239 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.003s sys 0.000s
I20260809 22:59:27.437537 32471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:27.437964 32239 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:27.438417 32239 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:59:27.438573 32239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal
uuid: "bae2d3a95a0f4da695a51f5be350262f"
format_stamp: "Formatted at 2026-08-09 22:59:27 on dist-test-slave-vxj2"
server_key: "f24535cf0a7fc83bbe4a2d3cfbcbd638"
server_key_iv: "6e87e080f48a603e8d8b9003090af938"
server_key_version: "encryptionkey@0"
I20260809 22:59:27.443532 32239 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:27.450707 32239 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 22:59:27.450979 32239 log.cc:1196] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 22:59:27.451615 32239 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Deleting consensus metadata
I20260809 22:59:27.451942 32239 master_runner.cc:315] Copying system tablet from 127.24.31.254:46425
I20260809 22:59:27.454681 32239 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: tablet copy: Beginning tablet copy session from remote peer at address 127.24.31.254:46425
I20260809 22:59:27.475103 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.455638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53820 (local address 127.24.31.254:46425)
0809 22:59:27.455840 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.455847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.460205 (+ 4358us) server_negotiation.cc:407] Connection header received
0809 22:59:27.460434 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.460440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.460526 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.460638 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:27.461725 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.462910 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.464835 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.465564 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.465676 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.468417 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:27.468423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:27.468429 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:27.468469 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:27.472020 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.472849 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.472854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.472858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.473071 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:27.473565 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:27.473570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:27.473573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:27.473907 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:27.474055 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.474532 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.474741 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:59:27.475579 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.455564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:53820)
0809 22:59:27.460054 (+ 4490us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.460076 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.460283 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.460685 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.460690 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.461268 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:27.461556 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.461570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.463078 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.463084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.464644 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.464661 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.465940 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.465946 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.466135 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.466911 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:27.466942 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:27.468178 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:27.472157 (+ 3979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.472162 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.472166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.472722 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.473254 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:27.473259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:27.473263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:27.473440 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:27.474029 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:27.474034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:27.474252 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.474399 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.474725 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4349,"thread_start_us":4198,"threads_started":1}
I20260809 22:59:27.476459 31620 tablet_copy_service.cc:140] P 66318d8fd4694039bb253dc9c4f3640e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bae2d3a95a0f4da695a51f5be350262f ({username='oryx', principal='oryx/127.24.31.252@KRBTEST.COM'} at 127.0.0.1:53820)
I20260809 22:59:27.476636 31620 tablet_copy_service.cc:161] P 66318d8fd4694039bb253dc9c4f3640e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bae2d3a95a0f4da695a51f5be350262f at {username='oryx', principal='oryx/127.24.31.252@KRBTEST.COM'} at 127.0.0.1:53820: session id = bae2d3a95a0f4da695a51f5be350262f-00000000000000000000000000000000
I20260809 22:59:27.480479 31620 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 22:59:27.482733 32239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 22:59:27.486939 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 31665
I20260809 22:59:27.490073 32239 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: tablet copy: Starting download of 0 data blocks...
I20260809 22:59:27.490361 32239 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: tablet copy: Starting download of 1 WAL segments...
I20260809 22:59:27.492764 32239 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 22:59:27.497895 31620 tablet_copy_service.cc:342] P 66318d8fd4694039bb253dc9c4f3640e: Request end of tablet copy session bae2d3a95a0f4da695a51f5be350262f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.31.252@KRBTEST.COM'} at 127.0.0.1:53820
I20260809 22:59:27.498031 31620 tablet_copy_service.cc:433] P 66318d8fd4694039bb253dc9c4f3640e: ending tablet copy session bae2d3a95a0f4da695a51f5be350262f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bae2d3a95a0f4da695a51f5be350262f
I20260809 22:59:27.509996 32239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:27.513983 32478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:27.516774 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:36033
--local_ip_for_outbound_sockets=127.24.31.193
--tserver_master_addrs=127.24.31.254:46425
--webserver_port=35811
--webserver_interface=127.24.31.193
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab
--principal=oryx/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.31.254:46425,127.24.31.253:38021
--tserver_master_addrs=127.24.31.254:46425,127.24.31.253:38021,127.24.31.252:33965 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:59:27.523825 32239 server_base.cc:1034] running on GCE node
W20260809 22:59:27.523675 32481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:27.524418 32479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:27.524966 32239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:27.525841 32239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:27.527195 32239 hybrid_clock.cc:648] HybridClock initialized: now 1786316367527142 us; error 44 us; skew 500 ppm
I20260809 22:59:27.527464 32239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:27 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316367, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:27.529783 32239 init.cc:377] Logged in from keytab as oryx/127.24.31.252@KRBTEST.COM (short username oryx)
I20260809 22:59:27.531247 32239 webserver.cc:460] Webserver started at http://127.24.31.252:34369/ using document root <none> and password file <none>
I20260809 22:59:27.531646 32239 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:27.531776 32239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:27.535195 32239 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 22:59:27.537387 32493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:27.537866 32239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 22:59:27.538014 32239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal
uuid: "bae2d3a95a0f4da695a51f5be350262f"
format_stamp: "Formatted at 2026-08-09 22:59:27 on dist-test-slave-vxj2"
server_key: "f24535cf0a7fc83bbe4a2d3cfbcbd638"
server_key_iv: "6e87e080f48a603e8d8b9003090af938"
server_key_version: "encryptionkey@0"
I20260809 22:59:27.538213 32239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:27.567713 32239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:27.572325 32239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:27.610231 32239 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.252:33965
I20260809 22:59:27.610255 32551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.252:33965 every 8 connection(s)
I20260809 22:59:27.611308 32239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
I20260809 22:59:27.618618 32552 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:59:27.622638 32552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Bootstrap starting.
I20260809 22:59:27.640509 32552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:59:27.641112 32552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Bootstrap complete.
I20260809 22:59:27.642252 32552 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } }
I20260809 22:59:27.642555 32552 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bae2d3a95a0f4da695a51f5be350262f, State: Initialized, Role: LEARNER
I20260809 22:59:27.642819 32552 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } }
I20260809 22:59:27.645671 32557 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.645973 32552 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:27.646214 32557 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: This master's current role is: LEARNER
I20260809 22:59:27.665078 32571 catalog_manager.cc:1259] Loaded cluster ID: 7c1eb2004dfb4568ab7d7d49fe0c9d62
I20260809 22:59:27.665534 32571 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: loading cluster ID for follower catalog manager: success
I20260809 22:59:27.668675 32571 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: acquiring CA information for follower catalog manager: success
I20260809 22:59:27.670732 32571 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 22:59:27.826758 32572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.819029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:33965 (local address 127.0.0.1:40958)
0809 22:59:27.820146 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:27.820164 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:27.820299 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:27.820724 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:27.820727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:27.820747 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:27.820905 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.820912 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.822910 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.822915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.824580 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:27.824593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.825861 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.825867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:27.826065 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.826111 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 22:59:27.826215 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 22:59:27.826354 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"thread_start_us":896,"threads_started":1}
I20260809 22:59:27.826763 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:27.819262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40958 (local address 127.24.31.252:33965)
0809 22:59:27.819972 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:27.819980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:27.820260 (+ 280us) server_negotiation.cc:407] Connection header received
0809 22:59:27.820440 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:27.820447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:27.820555 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:27.820716 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:27.821056 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.822741 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.824768 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:27.825690 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:27.825835 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:27.826124 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:27.826239 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 22:59:27.826372 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":357,"threads_started":1}
I20260809 22:59:27.827677 32524 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 22:59:27.828120 32557 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66318d8fd4694039bb253dc9c4f3640e. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:27.828433 32557 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: This master's current role is: LEARNER
W20260809 22:59:27.910138 32486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:27.910466 32486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:27.910634 32486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:27.918390 32486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:27.918527 32486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:27.918680 32486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:27.928733 32486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab
--principal=oryx/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:36033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=35811
--tserver_master_addrs=127.24.31.254:46425,127.24.31.253:38021,127.24.31.252:33965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:27.930256 32486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:27.931592 32486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:27.945472 32579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:27.945428 32582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:27.945392 32580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:27.946322 32486 server_base.cc:1034] running on GCE node
I20260809 22:59:27.946941 32486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:27.947882 32486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:27.949096 32486 hybrid_clock.cc:648] HybridClock initialized: now 1786316367949041 us; error 50 us; skew 500 ppm
I20260809 22:59:27.949425 32486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:27 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316367, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:27.952374 32486 init.cc:377] Logged in from keytab as oryx/127.24.31.193@KRBTEST.COM (short username oryx)
I20260809 22:59:27.953897 32486 webserver.cc:460] Webserver started at http://127.24.31.193:35811/ using document root <none> and password file <none>
I20260809 22:59:27.954560 32486 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:27.954715 32486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:27.960273 32486 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:59:27.962965 32589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:27.964306 32486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:59:27.964505 32486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "68596e5ddb484442af4a90aaa1af1356"
format_stamp: "Formatted at 2026-08-09 22:59:24 on dist-test-slave-vxj2"
server_key: "94da196900b5e89ebe93fc43452483db"
server_key_iv: "b275d3173217dbbcac42701c973242eb"
server_key_version: "encryptionkey@0"
I20260809 22:59:27.965579 32486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:27.999014 32486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:28.003333 32486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:28.003623 32486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:28.004691 32486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:28.006229 32486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:28.006349 32486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:28.006488 32486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:28.006577 32486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:28.041652 32486 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:36033
I20260809 22:59:28.041694 32726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:36033 every 8 connection(s)
I20260809 22:59:28.043139 32486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:28.048250 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 32486
I20260809 22:59:28.048672 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 31819
Aug 09 22:59:28 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316367, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.252@KRBTEST.COM
I20260809 22:59:28.077452 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.049399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:52899 (local address 127.24.31.252:33965)
0809 22:59:28.049954 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:28.049962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:28.054109 (+ 4147us) server_negotiation.cc:407] Connection header received
0809 22:59:28.058419 (+ 4310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:28.058425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:28.058522 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:28.058642 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:28.060210 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.061001 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.062671 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.063077 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.063212 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.068428 (+ 5216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:28.068439 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:28.068458 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:28.068515 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:28.072976 (+ 4461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.073973 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.073981 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.073984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.074155 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.074792 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.074796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.074799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.075512 (+ 713us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:28.075749 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:28.076735 (+ 986us) server_negotiation.cc:299] Negotiation successful
0809 22:59:28.077030 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400}
I20260809 22:59:28.079250 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.046995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:33965 (local address 127.24.31.193:52899)
0809 22:59:28.053919 (+ 6924us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:28.053987 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:28.058099 (+ 4112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:28.058898 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:28.058906 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:28.059455 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:28.060033 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.060052 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.061183 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.061188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.062515 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.062528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.063322 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.063329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.063583 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.064290 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:28.064319 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:28.068073 (+ 3754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:28.073171 (+ 5098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.073186 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.073207 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.073832 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.074312 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.074322 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.074327 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.074629 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.075842 (+ 1213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:28.075847 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:28.076273 (+ 426us) client_negotiation.cc:769] Sending connection context
0809 22:59:28.076597 (+ 324us) client_negotiation.cc:240] Negotiation successful
0809 22:59:28.077501 (+ 904us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6532,"thread_start_us":6266,"threads_started":1}
I20260809 22:59:28.081238 32727 heartbeater.cc:347] Connected to a master server at 127.24.31.252:33965
I20260809 22:59:28.081960 32727 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:28.083606 32727 heartbeater.cc:511] Master 127.24.31.252:33965 requested a full tablet report, sending...
Aug 09 22:59:28 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316367, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
Aug 09 22:59:28 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316367, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.253@KRBTEST.COM
I20260809 22:59:28.086454 32512 ts_manager.cc:194] Registered new tserver with Master: 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033)
Aug 09 22:59:28 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.194@KRBTEST.COM
I20260809 22:59:28.093312 32733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.046436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:56853 (local address 127.24.31.254:46425)
0809 22:59:28.058147 (+ 11711us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:28.058153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:28.072729 (+ 14576us) server_negotiation.cc:407] Connection header received
0809 22:59:28.072875 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:28.072879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:28.072955 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:28.073950 (+ 995us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:28.075688 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.076534 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.079675 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.080329 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.080430 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.086081 (+ 5651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:28.086088 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:28.086092 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:28.086133 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:28.089975 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.090854 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.090859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.090863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.091001 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.091633 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.091638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.091642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.092010 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:28.092172 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:28.092650 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 22:59:28.092887 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11578,"thread_start_us":11446,"threads_started":1}
I20260809 22:59:28.093371 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.050110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.193:56853)
0809 22:59:28.072597 (+ 22487us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:28.072614 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:28.072747 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:28.074175 (+ 1428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:28.074181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:28.075047 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:28.075541 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.075558 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.077665 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.077669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.079502 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.079520 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.080727 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.080733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.080920 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.081738 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:28.081763 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:28.085852 (+ 4089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:28.090144 (+ 4292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.090149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.090153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.090691 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.091383 (+ 692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.091386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.091388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.091513 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.092145 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:28.092152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:28.092315 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 22:59:28.092508 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 22:59:28.092833 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22361,"mutex_wait_us":67,"spinlock_wait_cycles":54272,"thread_start_us":22182,"threads_started":1}
I20260809 22:59:28.093467 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.049749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.24.31.193:58963)
0809 22:59:28.072189 (+ 22440us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:28.072214 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:28.072385 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:28.074192 (+ 1807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:28.074197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:28.075097 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:28.076838 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.076852 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.078256 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.078261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.079769 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.079792 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.080973 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.080979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.082453 (+ 1474us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.083082 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:28.083105 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:28.085874 (+ 2769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:28.090326 (+ 4452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.090333 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.090340 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.090990 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.091402 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.091407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.091411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.091610 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.092262 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:28.092265 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:28.092388 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 22:59:28.092537 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 22:59:28.092985 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22258,"mutex_wait_us":428,"thread_start_us":22000,"threads_started":1}
I20260809 22:59:28.094260 32728 heartbeater.cc:347] Connected to a master server at 127.24.31.254:46425
I20260809 22:59:28.094408 32728 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:28.094623 32732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.046306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:58963 (local address 127.24.31.253:38021)
0809 22:59:28.073503 (+ 27197us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:28.073510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:28.073537 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:59:28.073600 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:28.073605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:28.073695 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:28.073810 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:28.077027 (+ 3217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.078103 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.079934 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.080554 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.080679 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.086097 (+ 5418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:28.086104 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:28.086108 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:28.086147 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:28.090177 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.091124 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.091129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.091133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.091265 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.091737 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.091741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.091744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.092064 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:28.092185 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:28.093970 (+ 1785us) server_negotiation.cc:299] Negotiation successful
0809 22:59:28.094213 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27060,"thread_start_us":27689,"threads_started":1}
I20260809 22:59:28.094939 32728 heartbeater.cc:511] Master 127.24.31.254:46425 requested a full tablet report, sending...
I20260809 22:59:28.095878 32729 heartbeater.cc:347] Connected to a master server at 127.24.31.253:38021
I20260809 22:59:28.095996 32729 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:28.096226 31598 ts_manager.cc:194] Re-registered known tserver with Master: 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033)
I20260809 22:59:28.096333 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.049891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52100 (local address 127.24.31.194:45727)
0809 22:59:28.057715 (+ 7824us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:28.057726 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:28.071233 (+ 13507us) server_negotiation.cc:407] Connection header received
0809 22:59:28.073197 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:28.073212 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:28.073463 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:28.073636 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:28.075934 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.077454 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.080924 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.081928 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.082055 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.088288 (+ 6233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:28.088300 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:28.088312 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:28.088354 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:28.091861 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.092735 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.092746 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.092749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.092912 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.094046 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.094051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.094055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.094521 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:28.094823 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:28.095657 (+ 834us) server_negotiation.cc:299] Negotiation successful
0809 22:59:28.095931 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7599,"thread_start_us":7452,"threads_started":1}
I20260809 22:59:28.096401 32729 heartbeater.cc:511] Master 127.24.31.253:38021 requested a full tablet report, sending...
I20260809 22:59:28.097385 31598 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:56853
I20260809 22:59:28.097683 31962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:28.098299 32292 ts_manager.cc:194] Registered new tserver with Master: 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033)
I20260809 22:59:28.183979 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 31819
I20260809 22:59:28.204797 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:45727
--local_ip_for_outbound_sockets=127.24.31.194
--tserver_master_addrs=127.24.31.254:46425
--webserver_port=46673
--webserver_interface=127.24.31.194
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab
--principal=oryx/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.31.254:46425,127.24.31.253:38021
--tserver_master_addrs=127.24.31.254:46425,127.24.31.253:38021,127.24.31.252:33965 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:59:28.236938 32456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: NON_VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: true } } }
I20260809 22:59:28.237907 32456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:28.238618 32456 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: attempting to promote NON_VOTER bae2d3a95a0f4da695a51f5be350262f to VOTER
I20260809 22:59:28.239379 32456 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } }
I20260809 22:59:28.241274 32524 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 LEARNER]: Refusing update from remote peer 66318d8fd4694039bb253dc9c4f3640e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 22:59:28.241727 32302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Refusing update from remote peer 66318d8fd4694039bb253dc9c4f3640e: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 22:59:28.241874 32742 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 22:59:28.242424 32743 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [LEADER]: Connected to new peer: Peer: permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 22:59:28.245576 32742 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bae2d3a95a0f4da695a51f5be350262f (127.24.31.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:28.246150 32524 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bae2d3a95a0f4da695a51f5be350262f (127.24.31.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:28.246212 32302 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, bae2d3a95a0f4da695a51f5be350262f (127.24.31.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:28.249717 32742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:28.250108 32742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:28.249989 32456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:28.250545 32456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:28.250468 32742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:28.250999 32742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:28.253471 32557 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:28.253750 32557 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:59:28.258391 32748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66318d8fd4694039bb253dc9c4f3640e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:28.258862 32748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 22:59:28.555979 32741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:28.556245 32741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:28.556427 32741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:28.564010 32741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:28.564132 32741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:28.564342 32741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:28.574455 32741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab
--principal=oryx/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:45727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=46673
--tserver_master_addrs=127.24.31.254:46425,127.24.31.253:38021,127.24.31.252:33965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:28.575909 32741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:28.577426 32741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:28.591336 32753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:28.591367 32754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:28.591589 32756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:28.592217 32741 server_base.cc:1034] running on GCE node
I20260809 22:59:28.592836 32741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:28.594094 32741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:28.595337 32741 hybrid_clock.cc:648] HybridClock initialized: now 1786316368595289 us; error 34 us; skew 500 ppm
I20260809 22:59:28.595651 32741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:28 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316368, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:28.599025 32741 init.cc:377] Logged in from keytab as oryx/127.24.31.194@KRBTEST.COM (short username oryx)
I20260809 22:59:28.600656 32741 webserver.cc:460] Webserver started at http://127.24.31.194:46673/ using document root <none> and password file <none>
I20260809 22:59:28.601401 32741 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:28.601557 32741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:28.607528 32741 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:59:28.610387 32763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:28.612010 32741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:28.612159 32741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "a6df5178167940189db0b8836ca61bbe"
format_stamp: "Formatted at 2026-08-09 22:59:25 on dist-test-slave-vxj2"
server_key: "ca16e865f6074eb2f2d60cff5ef632f6"
server_key_iv: "1dd1f3540fbccd9ce210f69f9fbd068b"
server_key_version: "encryptionkey@0"
I20260809 22:59:28.613492 32741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:28.627586 32741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:28.631543 32741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:28.631785 32741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:28.632704 32741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:28.634179 32741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:28.634267 32741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:28.634346 32741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:28.634400 32741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:28.667574 32741 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:45727
I20260809 22:59:28.667661 431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:45727 every 8 connection(s)
I20260809 22:59:28.669132 32741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:28.678728 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 32741
I20260809 22:59:28.679092 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 31974
Aug 09 22:59:28 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.195@KRBTEST.COM
Aug 09 22:59:28 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316368, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.253@KRBTEST.COM
Aug 09 22:59:28 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316368, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
Aug 09 22:59:28 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316368, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.252@KRBTEST.COM
I20260809 22:59:28.714522 441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.674452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:45921 (local address 127.24.31.253:38021)
0809 22:59:28.692881 (+ 18429us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:28.692889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:28.693526 (+ 637us) server_negotiation.cc:407] Connection header received
0809 22:59:28.693713 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:28.693718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:28.693845 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:28.693962 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:28.695968 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.697207 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.700325 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.700966 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.701066 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.706548 (+ 5482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:28.706554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:28.706559 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:28.706625 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:28.710378 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.711919 (+ 1541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.711927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.711931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.712098 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.712605 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.712612 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.712617 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.713002 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:28.713135 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:28.713902 (+ 767us) server_negotiation.cc:299] Negotiation successful
0809 22:59:28.714132 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18282,"thread_start_us":18116,"threads_started":1}
I20260809 22:59:28.714555 436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.672213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:43505 (local address 127.24.31.252:33965)
0809 22:59:28.673013 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:28.673018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:28.684773 (+ 11755us) server_negotiation.cc:407] Connection header received
0809 22:59:28.693796 (+ 9023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:28.693802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:28.693897 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:28.693980 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:28.695974 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.697221 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.700322 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.701060 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.701213 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.707259 (+ 6046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:28.707265 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:28.707269 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:28.707316 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:28.711148 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.712053 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.712058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.712062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.712212 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.712670 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.712677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.712680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.713015 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:28.713127 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:28.713896 (+ 769us) server_negotiation.cc:299] Negotiation successful
0809 22:59:28.714144 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":550,"threads_started":1}
I20260809 22:59:28.714577 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.673394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:52739 (local address 127.24.31.254:46425)
0809 22:59:28.692610 (+ 19216us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:28.692617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:28.692638 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:59:28.692713 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:28.692721 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:28.692833 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:28.692968 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:28.696002 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.697129 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.700338 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.700977 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.701066 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.706818 (+ 5752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:28.706825 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:28.706829 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:28.706874 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:28.710534 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.711531 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.711537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.711540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.711700 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.712252 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.712258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.712261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.712656 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:28.712788 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:28.713899 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0809 22:59:28.714166 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19048,"thread_start_us":18933,"threads_started":1}
I20260809 22:59:28.716311 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.680788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58422 (local address 127.24.31.195:43257)
0809 22:59:28.688851 (+ 8063us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:28.688864 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:28.690340 (+ 1476us) server_negotiation.cc:407] Connection header received
0809 22:59:28.690565 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:28.690579 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:28.690902 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:28.691108 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:28.694001 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.695611 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.697545 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.698399 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.698516 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.707725 (+ 9209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:28.707747 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:28.707766 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:28.707829 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:28.712599 (+ 4770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.713529 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.713539 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.713543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.713723 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:28.714208 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:28.714213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:28.714216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:28.714646 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:28.714965 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:28.715598 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 22:59:28.715872 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7852,"thread_start_us":7696,"threads_started":1}
I20260809 22:59:28.717059 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.673795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.24.31.194:45921)
0809 22:59:28.693416 (+ 19621us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:28.693435 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:28.693578 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:28.694225 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:28.694229 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:28.694913 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:28.695786 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.695802 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.697377 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.697382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.700162 (+ 2780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.700170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.701507 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.701515 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.702006 (+ 491us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.702983 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:28.703022 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:28.706267 (+ 3245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:28.710552 (+ 4285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.710561 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.710580 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.711773 (+ 1193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.712253 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.712261 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.712264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.712456 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.713250 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:28.713255 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:28.713496 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 22:59:28.714837 (+ 1341us) client_negotiation.cc:240] Negotiation successful
0809 22:59:28.715511 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19475,"mutex_wait_us":806,"spinlock_wait_cycles":59136,"thread_start_us":19268,"threads_started":1}
I20260809 22:59:28.717100 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.672976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:33965 (local address 127.24.31.194:43505)
0809 22:59:28.684557 (+ 11581us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:28.684630 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:28.689684 (+ 5054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:28.694249 (+ 4565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:28.694255 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:28.694955 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:28.695773 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.695799 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.697405 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.697409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.700141 (+ 2732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.700159 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.701584 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.701592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.701999 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.702917 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:28.702958 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:28.707013 (+ 4055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:28.711288 (+ 4275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.711295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.711299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.711859 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.712362 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.712368 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.712372 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.712540 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.713250 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:28.713255 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:28.713496 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 22:59:28.714837 (+ 1341us) client_negotiation.cc:240] Negotiation successful
0809 22:59:28.715635 (+ 798us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11171,"mutex_wait_us":690,"spinlock_wait_cycles":768,"thread_start_us":20036,"threads_started":1}
I20260809 22:59:28.717624 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:28.673580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.194:52739)
0809 22:59:28.691832 (+ 18252us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:28.691856 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:28.691984 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:28.693536 (+ 1552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:28.693549 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:28.694676 (+ 1127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:28.695776 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.695802 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.697377 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.697382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.700154 (+ 2772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:28.700169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:28.701507 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:28.701515 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:28.702008 (+ 493us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:28.703072 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:28.703096 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:28.706594 (+ 3498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:28.710675 (+ 4081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.710681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.710684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.711381 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.711857 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:28.711862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:28.711866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:28.712106 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:28.712900 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:28.712908 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:28.713374 (+ 466us) client_negotiation.cc:769] Sending connection context
0809 22:59:28.715428 (+ 2054us) client_negotiation.cc:240] Negotiation successful
0809 22:59:28.715782 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18093,"mutex_wait_us":482,"thread_start_us":10395,"threads_started":1}
I20260809 22:59:28.718559 32118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:28.718984 433 heartbeater.cc:347] Connected to a master server at 127.24.31.254:46425
I20260809 22:59:28.719020 434 heartbeater.cc:347] Connected to a master server at 127.24.31.253:38021
I20260809 22:59:28.719077 432 heartbeater.cc:347] Connected to a master server at 127.24.31.252:33965
I20260809 22:59:28.719352 432 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:28.719372 433 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:28.719380 434 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:28.720362 432 heartbeater.cc:511] Master 127.24.31.252:33965 requested a full tablet report, sending...
I20260809 22:59:28.720539 434 heartbeater.cc:511] Master 127.24.31.253:38021 requested a full tablet report, sending...
I20260809 22:59:28.721527 433 heartbeater.cc:511] Master 127.24.31.254:46425 requested a full tablet report, sending...
I20260809 22:59:28.722576 31598 ts_manager.cc:194] Re-registered known tserver with Master: a6df5178167940189db0b8836ca61bbe (127.24.31.194:45727)
I20260809 22:59:28.722589 32512 ts_manager.cc:194] Registered new tserver with Master: a6df5178167940189db0b8836ca61bbe (127.24.31.194:45727)
I20260809 22:59:28.723511 31598 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:52739
I20260809 22:59:28.723577 32292 ts_manager.cc:194] Registered new tserver with Master: a6df5178167940189db0b8836ca61bbe (127.24.31.194:45727)
I20260809 22:59:28.810956 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 31974
I20260809 22:59:28.831476 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:43257
--local_ip_for_outbound_sockets=127.24.31.195
--tserver_master_addrs=127.24.31.254:46425
--webserver_port=43811
--webserver_interface=127.24.31.195
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab
--principal=oryx/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.31.254:46425,127.24.31.253:38021
--tserver_master_addrs=127.24.31.254:46425,127.24.31.253:38021,127.24.31.252:33965 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:59:29.100350 32728 heartbeater.cc:503] Master 127.24.31.254:46425 was elected leader, sending a full tablet report...
W20260809 22:59:29.169214 445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:29.169538 445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:29.169714 445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:29.177148 445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:29.177306 445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:29.177486 445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:29.187589 445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab
--principal=oryx/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:43257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=43811
--tserver_master_addrs=127.24.31.254:46425,127.24.31.253:38021,127.24.31.252:33965
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:29.188889 445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:29.190276 445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:29.203965 450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:29.204007 451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:29.204007 453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:29.204967 445 server_base.cc:1034] running on GCE node
I20260809 22:59:29.205605 445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:29.206580 445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:29.207767 445 hybrid_clock.cc:648] HybridClock initialized: now 1786316369207706 us; error 55 us; skew 500 ppm
I20260809 22:59:29.208081 445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:29 dist-test-slave-vxj2 krb5kdc[31549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316369, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:29.211254 445 init.cc:377] Logged in from keytab as oryx/127.24.31.195@KRBTEST.COM (short username oryx)
I20260809 22:59:29.212823 445 webserver.cc:460] Webserver started at http://127.24.31.195:43811/ using document root <none> and password file <none>
I20260809 22:59:29.213574 445 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:29.213730 445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:29.219324 445 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 22:59:29.222078 463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:29.223407 445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:29.223567 445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "d25cc28128cc4c5093710787c012491a"
format_stamp: "Formatted at 2026-08-09 22:59:25 on dist-test-slave-vxj2"
server_key: "ed4769e57b2437a88af59e98f67877c2"
server_key_iv: "522436599e5593395a1d7237a1e9fdd4"
server_key_version: "encryptionkey@0"
I20260809 22:59:29.224541 445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:29.245646 445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:29.250023 445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:29.250341 445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:29.251371 445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:29.252941 445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:29.253070 445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:29.253233 445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:29.253333 445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:29.292905 445 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:43257
I20260809 22:59:29.292963 597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:43257 every 8 connection(s)
I20260809 22:59:29.294492 445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:29.295516 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 445
Aug 09 22:59:29 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316369, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.253@KRBTEST.COM
Aug 09 22:59:29 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316369, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
Aug 09 22:59:29 dist-test-slave-vxj2 krb5kdc[31549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316369, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.252@KRBTEST.COM
I20260809 22:59:29.346133 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.303613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:57789 (local address 127.24.31.253:38021)
0809 22:59:29.307446 (+ 3833us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.307456 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.310702 (+ 3246us) server_negotiation.cc:407] Connection header received
0809 22:59:29.310833 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.310842 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.310936 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.311065 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.312576 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.313697 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.316445 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.317435 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.317578 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.325650 (+ 8072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.325661 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.325670 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.325737 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.330802 (+ 5065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.335031 (+ 4229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.335040 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.335047 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.335260 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.336841 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.336850 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.336856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.337376 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.337542 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.345372 (+ 7830us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.345634 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3662,"thread_start_us":3496,"threads_started":1}
I20260809 22:59:29.351552 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.298572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:33965 (local address 127.24.31.195:48497)
0809 22:59:29.299769 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:29.299841 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:29.303176 (+ 3335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:29.308432 (+ 5256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:29.308450 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:29.311164 (+ 2714us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:29.311506 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.311524 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.313089 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.313095 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.320722 (+ 7627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.320743 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.322550 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.322559 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.322787 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.323887 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:29.323918 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:29.340952 (+ 17034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:29.347681 (+ 6729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:29.347687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:29.347694 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:29.348220 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:29.348686 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:29.348694 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:29.348700 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:29.348890 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:29.349746 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:29.349754 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:29.349965 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 22:59:29.350124 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 22:59:29.350478 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":3,"thread_start_us":1117,"threads_started":1}
I20260809 22:59:29.353941 609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.303117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.24.31.195:57789)
0809 22:59:29.310392 (+ 7275us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:29.310414 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:29.310551 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:29.311331 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:29.311336 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:29.311965 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:29.312364 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.312379 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.313878 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.313885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.316192 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.316216 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.318035 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.318047 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.318483 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.321089 (+ 2606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:29.321130 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:29.325315 (+ 4185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:29.333851 (+ 8536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:29.333865 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:29.333889 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:29.334849 (+ 960us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:29.336372 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:29.336380 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:29.336386 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:29.336680 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:29.342635 (+ 5955us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:29.342644 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:29.343139 (+ 495us) client_negotiation.cc:769] Sending connection context
0809 22:59:29.343568 (+ 429us) client_negotiation.cc:240] Negotiation successful
0809 22:59:29.344213 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7132,"thread_start_us":6948,"threads_started":1}
I20260809 22:59:29.354362 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.299298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:46457 (local address 127.24.31.254:46425)
0809 22:59:29.308423 (+ 9125us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.308430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.308452 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:59:29.308516 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.308523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.308602 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.308733 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.311091 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.312108 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.325664 (+ 13556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.326656 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.326821 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.336477 (+ 9656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.336487 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.336495 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.336552 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.341382 (+ 4830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.347629 (+ 6247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.347636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.347640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.347821 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.352354 (+ 4533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.352362 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.352368 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.352837 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.353000 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.353682 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.353968 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":8570,"thread_start_us":8480,"threads_started":1}
I20260809 22:59:29.354985 606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.299234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:48497 (local address 127.24.31.252:33965)
0809 22:59:29.301330 (+ 2096us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.301338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.301362 (+ 24us) server_negotiation.cc:407] Connection header received
0809 22:59:29.307600 (+ 6238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.307611 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.307732 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.309599 (+ 1867us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.311731 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.312893 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.320929 (+ 8036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.321632 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.321795 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.342987 (+ 21192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.342995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.343000 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.343048 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.347085 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.348378 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.348384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.348388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.348545 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.349044 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.349051 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.349054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.349483 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.349641 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.354522 (+ 4881us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.354732 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1884,"thread_start_us":1766,"threads_started":1}
I20260809 22:59:29.355525 603 heartbeater.cc:347] Connected to a master server at 127.24.31.253:38021
I20260809 22:59:29.355888 603 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:29.356057 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.298573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.195:46457)
0809 22:59:29.304055 (+ 5482us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:29.304079 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:29.304219 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:29.308755 (+ 4536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:29.308759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:29.309631 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:29.310892 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.310923 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.312285 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.312290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.323288 (+ 10998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.323311 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.331092 (+ 7781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.331103 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.331399 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.332080 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:29.332113 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:29.336165 (+ 4052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:29.346831 (+ 10666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:29.346840 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:29.346844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:29.347457 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:29.351990 (+ 4533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:29.351996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:29.352000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:29.352170 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:29.353106 (+ 936us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:29.353113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:29.353375 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 22:59:29.353531 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 22:59:29.353852 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5295,"mutex_wait_us":3391,"thread_start_us":4264,"threads_started":1}
I20260809 22:59:29.356984 602 heartbeater.cc:347] Connected to a master server at 127.24.31.254:46425
I20260809 22:59:29.357102 600 heartbeater.cc:347] Connected to a master server at 127.24.31.252:33965
I20260809 22:59:29.357290 600 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:29.357846 600 heartbeater.cc:511] Master 127.24.31.252:33965 requested a full tablet report, sending...
I20260809 22:59:29.357142 602 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:29.358585 602 heartbeater.cc:511] Master 127.24.31.254:46425 requested a full tablet report, sending...
I20260809 22:59:29.359141 603 heartbeater.cc:511] Master 127.24.31.253:38021 requested a full tablet report, sending...
I20260809 22:59:29.359993 32512 ts_manager.cc:194] Registered new tserver with Master: d25cc28128cc4c5093710787c012491a (127.24.31.195:43257)
I20260809 22:59:29.360186 31598 ts_manager.cc:194] Re-registered known tserver with Master: d25cc28128cc4c5093710787c012491a (127.24.31.195:43257)
I20260809 22:59:29.361296 31598 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:46457
I20260809 22:59:29.362072 32292 ts_manager.cc:194] Registered new tserver with Master: d25cc28128cc4c5093710787c012491a (127.24.31.195:43257)
I20260809 22:59:29.373090 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.325182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59962 (local address 127.24.31.253:38021)
0809 22:59:29.346748 (+ 21566us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.346756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.353099 (+ 6343us) server_negotiation.cc:407] Connection header received
0809 22:59:29.353214 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.353219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.353334 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.353442 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.358589 (+ 5147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.359661 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.361618 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.362432 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.362562 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.366683 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.366688 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.366691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.366721 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.370093 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.370924 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.370930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.370934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.371104 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.371590 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.371596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.371602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.371886 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.372180 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.372490 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.372698 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21443,"thread_start_us":22271,"threads_started":1}
I20260809 22:59:29.375792 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.314895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.24.31.252:33965)
0809 22:59:29.323860 (+ 8965us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.323867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.348355 (+ 24488us) server_negotiation.cc:407] Connection header received
0809 22:59:29.348407 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.348411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.348492 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.348607 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.351522 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.360205 (+ 8683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.364142 (+ 3937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.364595 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.364766 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.369007 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.369015 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.369020 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.369062 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.372899 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.373634 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.373643 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.373647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.373817 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.374318 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.374323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.374327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.374659 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.374833 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.375201 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.375424 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8826,"thread_start_us":9739,"threads_started":1}
I20260809 22:59:29.377059 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.313963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.24.31.254:46425)
0809 22:59:29.353357 (+ 39394us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.353361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.353376 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:59:29.353414 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.353417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.353477 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.353617 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.358048 (+ 4431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.359529 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.364164 (+ 4635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.364941 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.365086 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.369369 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.369376 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.369381 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.369451 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.373338 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.374904 (+ 1566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.374908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.374910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.375037 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.375445 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.375451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.375455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.375742 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.376034 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.376399 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.376630 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":474,"server-negotiator.queue_time_us":39263,"thread_start_us":39113,"threads_started":1}
I20260809 22:59:29.381812 31598 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:59:29.391117 31598 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:59:29.429548 639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.417034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:45727 (local address 127.0.0.1:52114)
0809 22:59:29.417764 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:29.417784 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:29.417910 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:29.419098 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:29.419103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:29.419130 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:29.419574 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.419590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.422587 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.422596 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.425571 (+ 2975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.425593 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.428248 (+ 2655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.428257 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.428522 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.428580 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 22:59:29.428666 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 22:59:29.428836 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":444,"threads_started":1}
I20260809 22:59:29.431887 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.417235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.24.31.194:45727)
0809 22:59:29.418404 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.418413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.418455 (+ 42us) server_negotiation.cc:407] Connection header received
0809 22:59:29.418573 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.418581 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.418867 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.420030 (+ 1163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:29.420285 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.422359 (+ 2074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.425918 (+ 3559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.427944 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.430810 (+ 2866us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.431129 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.431224 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.431478 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":1058,"threads_started":1}
I20260809 22:59:29.436180 358 tablet_service.cc:1467] Processing CreateTablet for tablet e8956eedf7404e7a8cc4647cb1458d41 (DEFAULT_TABLE table=test-table [id=1f21c71157a5495996269b8d4164f54f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:29.437964 358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8956eedf7404e7a8cc4647cb1458d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:29.440654 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.418929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33362 (local address 127.24.31.193:36033)
0809 22:59:29.432542 (+ 13613us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.432552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.432581 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:59:29.432693 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.432701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.432965 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.433207 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:29.434190 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.435669 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.438413 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.439460 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.439615 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.439781 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.440202 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.440370 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13411,"thread_start_us":13311,"threads_started":1}
I20260809 22:59:29.440706 641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.418378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33362)
0809 22:59:29.418918 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:29.418937 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:29.419058 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:29.433273 (+ 14215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:29.433282 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:29.433331 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:29.433929 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.433949 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.435845 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.435850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.438216 (+ 2366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.438231 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.439901 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.439906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.440028 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.440051 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:59:29.440155 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 22:59:29.440311 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"spinlock_wait_cycles":3840,"thread_start_us":377,"threads_started":1}
I20260809 22:59:29.445231 32647 tablet_service.cc:1467] Processing CreateTablet for tablet e8956eedf7404e7a8cc4647cb1458d41 (DEFAULT_TABLE table=test-table [id=1f21c71157a5495996269b8d4164f54f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:29.446930 32647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8956eedf7404e7a8cc4647cb1458d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:29.452814 639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.419820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:43257 (local address 127.0.0.1:58430)
0809 22:59:29.429841 (+ 10021us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:29.429860 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:29.429954 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:29.445841 (+ 15887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:29.445846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:29.445874 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:29.446037 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.446048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.448231 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.448236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.450210 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.450226 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.452015 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.452021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.452223 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.452268 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 22:59:29.452365 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 22:59:29.452515 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9924,"thread_start_us":17615,"threads_started":1}
I20260809 22:59:29.452912 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.419984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58430 (local address 127.24.31.195:43257)
0809 22:59:29.445091 (+ 25107us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.445100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.445136 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:59:29.445282 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.445291 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.445537 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.445814 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:29.446249 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.448034 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.450392 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.451855 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.451985 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.452190 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.452418 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.452582 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24923,"thread_start_us":25234,"threads_started":1}
I20260809 22:59:29.456019 646 tablet_bootstrap.cc:492] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: Bootstrap starting.
I20260809 22:59:29.456858 524 tablet_service.cc:1467] Processing CreateTablet for tablet e8956eedf7404e7a8cc4647cb1458d41 (DEFAULT_TABLE table=test-table [id=1f21c71157a5495996269b8d4164f54f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:29.458319 524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8956eedf7404e7a8cc4647cb1458d41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:29.459913 646 tablet_bootstrap.cc:654] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:29.461302 646 log.cc:824] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:29.461841 647 tablet_bootstrap.cc:492] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: Bootstrap starting.
I20260809 22:59:29.465370 647 tablet_bootstrap.cc:654] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:29.466691 647 log.cc:824] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:29.467548 646 tablet_bootstrap.cc:492] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: No bootstrap required, opened a new log
I20260809 22:59:29.467870 646 ts_tablet_manager.cc:1397] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260809 22:59:29.470567 647 tablet_bootstrap.cc:492] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: No bootstrap required, opened a new log
I20260809 22:59:29.470888 647 ts_tablet_manager.cc:1397] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260809 22:59:29.472563 650 tablet_bootstrap.cc:492] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: Bootstrap starting.
I20260809 22:59:29.475455 650 tablet_bootstrap.cc:654] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:29.476433 650 log.cc:824] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:29.478961 650 tablet_bootstrap.cc:492] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: No bootstrap required, opened a new log
I20260809 22:59:29.479216 650 ts_tablet_manager.cc:1397] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 22:59:29.479187 646 raft_consensus.cc:348] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:29.479650 646 raft_consensus.cc:374] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:29.479753 646 raft_consensus.cc:729] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6df5178167940189db0b8836ca61bbe, State: Initialized, Role: FOLLOWER
I20260809 22:59:29.480363 646 consensus_queue.cc:260] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:29.481600 647 raft_consensus.cc:348] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:29.482167 647 raft_consensus.cc:374] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:29.482318 647 raft_consensus.cc:729] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68596e5ddb484442af4a90aaa1af1356, State: Initialized, Role: FOLLOWER
I20260809 22:59:29.482496 646 ts_tablet_manager.cc:1428] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260809 22:59:29.482771 433 heartbeater.cc:503] Master 127.24.31.254:46425 was elected leader, sending a full tablet report...
I20260809 22:59:29.483783 647 consensus_queue.cc:260] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:29.490736 650 raft_consensus.cc:348] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:29.491201 650 raft_consensus.cc:374] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:29.491315 650 raft_consensus.cc:729] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25cc28128cc4c5093710787c012491a, State: Initialized, Role: FOLLOWER
I20260809 22:59:29.491842 647 ts_tablet_manager.cc:1428] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: Time spent starting tablet: real 0.021s user 0.010s sys 0.008s
I20260809 22:59:29.491873 650 consensus_queue.cc:260] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:29.493518 602 heartbeater.cc:503] Master 127.24.31.254:46425 was elected leader, sending a full tablet report...
I20260809 22:59:29.496747 650 ts_tablet_manager.cc:1428] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
W20260809 22:59:29.552548 32731 tablet.cc:2369] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:59:29.558853 611 tablet.cc:2369] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:59:29.566314 654 raft_consensus.cc:482] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:29.566644 654 raft_consensus.cc:504] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:29.568315 654 leader_election.cc:288] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6df5178167940189db0b8836ca61bbe (127.24.31.194:45727), 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033)
I20260809 22:59:29.574857 640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.568764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:33743 (local address 127.24.31.194:45727)
0809 22:59:29.569003 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.569010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.569109 (+ 99us) server_negotiation.cc:407] Connection header received
0809 22:59:29.569339 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.569344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.569425 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.569530 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:29.570093 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.570828 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.573006 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.573828 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.573992 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.574067 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.574423 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.574579 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 22:59:29.574867 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.568789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:35755 (local address 127.24.31.193:36033)
0809 22:59:29.569002 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.569008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.569109 (+ 101us) server_negotiation.cc:407] Connection header received
0809 22:59:29.569352 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.569356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.569425 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.569530 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:29.570049 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.570834 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.572979 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.573823 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.573989 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.574066 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.574434 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.574600 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 22:59:29.574962 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.568649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.24.31.195:35755)
0809 22:59:29.569004 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:29.569024 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:29.569188 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:29.569560 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:29.569565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:29.569603 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:29.569894 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.569903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.571075 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.571080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.572840 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.572852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.574006 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.574012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.574218 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.574266 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 22:59:29.574395 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 22:59:29.574600 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":223,"spinlock_wait_cycles":11392}
I20260809 22:59:29.575381 605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.568721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:45727 (local address 127.24.31.195:33743)
0809 22:59:29.568997 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:29.569025 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:29.569189 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:29.569560 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:29.569565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:29.569601 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:29.569881 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.569894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.571121 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.571127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.572859 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:29.572871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.573992 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.573999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:29.574217 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.574259 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 22:59:29.574391 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 22:59:29.574586 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":117,"spinlock_wait_cycles":12160}
I20260809 22:59:29.575891 32671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8956eedf7404e7a8cc4647cb1458d41" candidate_uuid: "d25cc28128cc4c5093710787c012491a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68596e5ddb484442af4a90aaa1af1356" is_pre_election: true
I20260809 22:59:29.576375 32671 raft_consensus.cc:2454] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d25cc28128cc4c5093710787c012491a in term 0.
I20260809 22:59:29.576332 381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8956eedf7404e7a8cc4647cb1458d41" candidate_uuid: "d25cc28128cc4c5093710787c012491a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6df5178167940189db0b8836ca61bbe" is_pre_election: true
I20260809 22:59:29.576820 381 raft_consensus.cc:2454] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d25cc28128cc4c5093710787c012491a in term 0.
I20260809 22:59:29.577026 467 leader_election.cc:302] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68596e5ddb484442af4a90aaa1af1356, d25cc28128cc4c5093710787c012491a; no voters:
I20260809 22:59:29.577456 654 raft_consensus.cc:2781] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:29.577616 654 raft_consensus.cc:482] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:29.577731 654 raft_consensus.cc:3037] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:29.580029 654 raft_consensus.cc:504] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:29.580570 654 leader_election.cc:288] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [CANDIDATE]: Term 1 election: Requested vote from peers a6df5178167940189db0b8836ca61bbe (127.24.31.194:45727), 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033)
I20260809 22:59:29.580993 381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8956eedf7404e7a8cc4647cb1458d41" candidate_uuid: "d25cc28128cc4c5093710787c012491a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6df5178167940189db0b8836ca61bbe"
I20260809 22:59:29.581022 32671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8956eedf7404e7a8cc4647cb1458d41" candidate_uuid: "d25cc28128cc4c5093710787c012491a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68596e5ddb484442af4a90aaa1af1356"
I20260809 22:59:29.581218 381 raft_consensus.cc:3037] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:29.581229 32671 raft_consensus.cc:3037] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:29.583549 32671 raft_consensus.cc:2454] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d25cc28128cc4c5093710787c012491a in term 1.
I20260809 22:59:29.583774 381 raft_consensus.cc:2454] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d25cc28128cc4c5093710787c012491a in term 1.
I20260809 22:59:29.583906 467 leader_election.cc:302] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68596e5ddb484442af4a90aaa1af1356, d25cc28128cc4c5093710787c012491a; no voters:
I20260809 22:59:29.584189 654 raft_consensus.cc:2781] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:29.584664 654 raft_consensus.cc:686] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 1 LEADER]: Becoming Leader. State: Replica: d25cc28128cc4c5093710787c012491a, State: Running, Role: LEADER
I20260809 22:59:29.585210 654 consensus_queue.cc:237] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:29.591539 31597 catalog_manager.cc:5495] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a reported cstate change: term changed from 0 to 1, leader changed from <none> to d25cc28128cc4c5093710787c012491a (127.24.31.195). New cstate: current_term: 1 leader_uuid: "d25cc28128cc4c5093710787c012491a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } health_report { overall_health: HEALTHY } } }
I20260809 22:59:29.621904 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.616678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58438 (local address 127.24.31.195:43257)
0809 22:59:29.616916 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.616925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.616945 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:59:29.617044 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.617048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.617123 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.617325 (+ 202us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:29.617691 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.618437 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.619734 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.620146 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.620294 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.620779 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:29.621347 (+ 568us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:29.621508 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.621560 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.621639 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:59:29.630084 32671 raft_consensus.cc:1261] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [term 1 FOLLOWER]: Refusing update from remote peer d25cc28128cc4c5093710787c012491a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:29.630124 381 raft_consensus.cc:1261] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [term 1 FOLLOWER]: Refusing update from remote peer d25cc28128cc4c5093710787c012491a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:29.631055 654 consensus_queue.cc:1035] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [LEADER]: Connected to new peer: Peer: permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:29.631418 655 consensus_queue.cc:1035] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:29.646625 658 mvcc.cc:205] Tried to move back new op lower bound from 7316751849995091968 to 7316751849824677888. Current Snapshot: MvccSnapshot[applied={T|T < 7316751849995091968}]
I20260809 22:59:29.648793 660 mvcc.cc:205] Tried to move back new op lower bound from 7316751849995091968 to 7316751849824677888. Current Snapshot: MvccSnapshot[applied={T|T < 7316751849995091968}]
I20260809 22:59:29.661232 31598 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:29.661515 31598 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:29.667460 31598 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 66318d8fd4694039bb253dc9c4f3640e: Sending DeleteTablet for 3 replicas of tablet e8956eedf7404e7a8cc4647cb1458d41
I20260809 22:59:29.669045 32647 tablet_service.cc:1514] Processing DeleteTablet for tablet e8956eedf7404e7a8cc4647cb1458d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:29 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:33362
I20260809 22:59:29.669270 358 tablet_service.cc:1514] Processing DeleteTablet for tablet e8956eedf7404e7a8cc4647cb1458d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:29 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:52114
I20260809 22:59:29.670518 524 tablet_service.cc:1514] Processing DeleteTablet for tablet e8956eedf7404e7a8cc4647cb1458d41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:29 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:58430
I20260809 22:59:29.680023 668 tablet_replica.cc:330] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: stopping tablet replica
I20260809 22:59:29.681811 668 raft_consensus.cc:2229] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:29.683851 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 31562
I20260809 22:59:29.684959 31630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:29.685351 668 raft_consensus.cc:2258] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:29.697135 668 ts_tablet_manager.cc:1905] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:29.703359 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.673472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59964 (local address 127.24.31.253:38021)
0809 22:59:29.673654 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.673660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.681136 (+ 7476us) server_negotiation.cc:407] Connection header received
0809 22:59:29.681248 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.681254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.681343 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.681476 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.687673 (+ 6197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.688895 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.691406 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.692131 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.692279 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.696703 (+ 4424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.696710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.696715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.696769 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.700350 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.701199 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.701205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.701209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.701376 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.701733 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.701738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.701741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.702132 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.702380 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.702738 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.703009 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:59:29.706035 668 ts_tablet_manager.cc:1918] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:29.706231 668 log.cc:1196] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/e8956eedf7404e7a8cc4647cb1458d41
I20260809 22:59:29.706991 668 ts_tablet_manager.cc:1939] T e8956eedf7404e7a8cc4647cb1458d41 P a6df5178167940189db0b8836ca61bbe: Deleting consensus metadata
I20260809 22:59:29.714610 669 tablet_replica.cc:330] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: stopping tablet replica
I20260809 22:59:29.715175 669 raft_consensus.cc:2229] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:59:29.722550 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.681879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40976 (local address 127.24.31.252:33965)
0809 22:59:29.682083 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.682091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.704348 (+ 22257us) server_negotiation.cc:407] Connection header received
0809 22:59:29.704438 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.704444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.704523 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.704617 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.706785 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.707935 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.709946 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.710670 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.710816 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.715563 (+ 4747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.715570 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.715575 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.715619 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.719268 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.720238 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.720244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.720248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.720398 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.720912 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.720917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.720921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.721315 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.721608 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.721988 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.722219 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:59:29.724107 667 tablet_replica.cc:330] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: stopping tablet replica
I20260809 22:59:29.724131 669 raft_consensus.cc:2258] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:29.725382 667 raft_consensus.cc:2229] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:29.726027 667 raft_consensus.cc:2258] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:29.726904 669 ts_tablet_manager.cc:1905] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:29.727921 667 ts_tablet_manager.cc:1905] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:29.734913 667 ts_tablet_manager.cc:1918] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:29.735095 667 log.cc:1196] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/e8956eedf7404e7a8cc4647cb1458d41
I20260809 22:59:29.735069 669 ts_tablet_manager.cc:1918] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:29.735239 669 log.cc:1196] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/e8956eedf7404e7a8cc4647cb1458d41
I20260809 22:59:29.735622 667 ts_tablet_manager.cc:1939] T e8956eedf7404e7a8cc4647cb1458d41 P 68596e5ddb484442af4a90aaa1af1356: Deleting consensus metadata
I20260809 22:59:29.735862 669 ts_tablet_manager.cc:1939] T e8956eedf7404e7a8cc4647cb1458d41 P d25cc28128cc4c5093710787c012491a: Deleting consensus metadata
I20260809 22:59:29.847141 31585 catalog_manager.cc:4841] TS 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033): tablet e8956eedf7404e7a8cc4647cb1458d41 (table test-table [id=1f21c71157a5495996269b8d4164f54f]) successfully deleted
I20260809 22:59:29.848588 31580 catalog_manager.cc:4841] TS a6df5178167940189db0b8836ca61bbe (127.24.31.194:45727): tablet e8956eedf7404e7a8cc4647cb1458d41 (table test-table [id=1f21c71157a5495996269b8d4164f54f]) successfully deleted
I20260809 22:59:29.848711 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 31562
I20260809 22:59:29.905138 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.888118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.24.31.252:33965)
0809 22:59:29.888322 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.888329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.888937 (+ 608us) server_negotiation.cc:407] Connection header received
0809 22:59:29.889205 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.889210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.889322 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.889418 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.891290 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.892210 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.894061 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.894501 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.894661 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.898709 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.898716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.898720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.898778 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.901977 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.902678 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.902683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.902685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.902815 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.903245 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.903250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.903253 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.903643 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.903810 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.904460 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.904680 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:59:29.906411 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:29.888482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59980 (local address 127.24.31.253:38021)
0809 22:59:29.888659 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:29.888665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:29.889602 (+ 937us) server_negotiation.cc:407] Connection header received
0809 22:59:29.889747 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:29.889753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:29.889833 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:29.889955 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:29.891907 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.892750 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.894162 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:29.894800 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:29.894919 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:29.899540 (+ 4621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:29.899546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:29.899550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:29.899592 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:29.903185 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.903994 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.904000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.904004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.904173 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:29.904613 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:29.904619 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:29.904622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:29.904995 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:29.905289 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:29.905667 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 22:59:29.905912 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 22:59:30.654024 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:30.653272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.194:33659)
0809 22:59:30.653742 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:30.653853 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":251,"threads_started":1}
I20260809 22:59:30.654059 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:30.653211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.195:56909)
0809 22:59:30.653726 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:30.653865 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":287,"threads_started":1}
W20260809 22:59:30.654479 602 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:46425 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
W20260809 22:59:30.654498 433 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:46425 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
I20260809 22:59:30.657917 681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:30.657141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.193:58767)
0809 22:59:30.657607 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:30.657771 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":224,"threads_started":1}
W20260809 22:59:30.658361 32728 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:46425 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
I20260809 22:59:31.168578 683 raft_consensus.cc:482] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 66318d8fd4694039bb253dc9c4f3640e)
I20260809 22:59:31.168974 683 raft_consensus.cc:504] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } }
I20260809 22:59:31.170728 683 leader_election.cc:288] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66318d8fd4694039bb253dc9c4f3640e (127.24.31.254:46425), 93cb005232ed48a78e2d5b8898761e45 (127.24.31.253:38021)
I20260809 22:59:31.172590 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.170878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54218)
0809 22:59:31.172030 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.172294 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":977,"thread_start_us":813,"threads_started":1}
W20260809 22:59:31.172930 32496 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
I20260809 22:59:31.173400 687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 66318d8fd4694039bb253dc9c4f3640e)
I20260809 22:59:31.173810 687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } }
I20260809 22:59:31.174714 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.174344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54224)
0809 22:59:31.174493 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.174588 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 22:59:31.174902 32496 leader_election.cc:334] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 66318d8fd4694039bb253dc9c4f3640e (127.24.31.254:46425): Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
I20260809 22:59:31.175506 687 leader_election.cc:288] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 66318d8fd4694039bb253dc9c4f3640e (127.24.31.254:46425), bae2d3a95a0f4da695a51f5be350262f (127.24.31.252:33965)
I20260809 22:59:31.183506 685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.171104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59988 (local address 127.24.31.253:38021)
0809 22:59:31.171856 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.171864 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.172423 (+ 559us) server_negotiation.cc:407] Connection header received
0809 22:59:31.172638 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.172644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.172755 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.172865 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:31.173769 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.175084 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.178748 (+ 3664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.181414 (+ 2666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.181562 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.181735 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.182647 (+ 912us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.182862 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":312,"threads_started":1}
I20260809 22:59:31.184562 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.171145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:38021 (local address 127.0.0.1:59988)
0809 22:59:31.172298 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.172316 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:31.172464 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:31.172954 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:31.172965 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:31.173026 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:31.173539 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.173560 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.175255 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.175260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.178453 (+ 3193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.178480 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.181958 (+ 3478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.181969 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.182328 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.182409 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 22:59:31.183764 (+ 1355us) client_negotiation.cc:240] Negotiation successful
0809 22:59:31.183973 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"thread_start_us":879,"threads_started":1}
I20260809 22:59:31.186124 32302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bae2d3a95a0f4da695a51f5be350262f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "93cb005232ed48a78e2d5b8898761e45" is_pre_election: true
I20260809 22:59:31.186460 32302 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bae2d3a95a0f4da695a51f5be350262f in term 1.
I20260809 22:59:31.187726 32498 leader_election.cc:302] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93cb005232ed48a78e2d5b8898761e45, bae2d3a95a0f4da695a51f5be350262f; no voters: 66318d8fd4694039bb253dc9c4f3640e
I20260809 22:59:31.188288 683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 22:59:31.188481 683 raft_consensus.cc:482] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 66318d8fd4694039bb253dc9c4f3640e)
I20260809 22:59:31.188624 683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:59:31.189179 690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.175650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54238)
0809 22:59:31.188594 (+ 12944us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.188843 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12813,"thread_start_us":13020,"threads_started":1}
W20260809 22:59:31.189536 32273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
I20260809 22:59:31.192102 683 raft_consensus.cc:504] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } }
I20260809 22:59:31.192998 683 leader_election.cc:288] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [CANDIDATE]: Term 2 election: Requested vote from peers 66318d8fd4694039bb253dc9c4f3640e (127.24.31.254:46425), 93cb005232ed48a78e2d5b8898761e45 (127.24.31.253:38021)
I20260809 22:59:31.193579 32302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bae2d3a95a0f4da695a51f5be350262f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "93cb005232ed48a78e2d5b8898761e45"
I20260809 22:59:31.193828 32302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:59:31.194007 690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.193648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54254)
0809 22:59:31.193784 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.193875 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:59:31.194018 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.193505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54252)
0809 22:59:31.193765 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.193863 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260809 22:59:31.194350 32273 leader_election.cc:334] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 66318d8fd4694039bb253dc9c4f3640e (127.24.31.254:46425): Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
I20260809 22:59:31.195164 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.194730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54256)
0809 22:59:31.194895 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.194986 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 22:59:31.195410 32496 leader_election.cc:334] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 66318d8fd4694039bb253dc9c4f3640e (127.24.31.254:46425): Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
I20260809 22:59:31.196820 32302 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bae2d3a95a0f4da695a51f5be350262f in term 2.
I20260809 22:59:31.197301 32498 leader_election.cc:302] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 93cb005232ed48a78e2d5b8898761e45, bae2d3a95a0f4da695a51f5be350262f; no voters: 66318d8fd4694039bb253dc9c4f3640e
I20260809 22:59:31.197589 683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:59:31.198143 683 raft_consensus.cc:686] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [term 2 LEADER]: Becoming Leader. State: Replica: bae2d3a95a0f4da695a51f5be350262f, State: Running, Role: LEADER
I20260809 22:59:31.198215 689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.175701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:33965 (local address 127.0.0.1:40994)
0809 22:59:31.188956 (+ 13255us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.188983 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:31.189098 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:31.190005 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:31.190476 (+ 471us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:31.190518 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:31.191021 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.191037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.192963 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.192972 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.195770 (+ 2798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.195785 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.197254 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.197262 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.197495 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.197533 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 22:59:31.197659 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 22:59:31.197805 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13132,"thread_start_us":12669,"threads_started":1}
I20260809 22:59:31.198530 691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.176091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.24.31.252:33965)
0809 22:59:31.189626 (+ 13535us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.189634 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.189670 (+ 36us) server_negotiation.cc:407] Connection header received
0809 22:59:31.189745 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.189751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.189837 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.189972 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:31.191294 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.192744 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.195944 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.196985 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.197157 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.197340 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.197687 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.197938 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13353,"thread_start_us":13572,"threads_started":1}
I20260809 22:59:31.198625 683 consensus_queue.cc:237] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } }
I20260809 22:59:31.199254 32524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "93cb005232ed48a78e2d5b8898761e45" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bae2d3a95a0f4da695a51f5be350262f" is_pre_election: true
I20260809 22:59:31.200028 32274 leader_election.cc:302] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93cb005232ed48a78e2d5b8898761e45; no voters: 66318d8fd4694039bb253dc9c4f3640e, bae2d3a95a0f4da695a51f5be350262f
I20260809 22:59:31.200525 687 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 22:59:31.200546 693 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bae2d3a95a0f4da695a51f5be350262f. Latest consensus state: current_term: 2 leader_uuid: "bae2d3a95a0f4da695a51f5be350262f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:31.200984 693 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:31.201773 696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:31.205916 696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:31.206889 696 catalog_manager.cc:1259] Loaded cluster ID: 7c1eb2004dfb4568ab7d7d49fe0c9d62
I20260809 22:59:31.207011 696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:31.208366 696 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:31.209403 696 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Loaded TSK: 0
I20260809 22:59:31.210340 696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 22:59:31.249500 32512 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:59:31.256544 32512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:59:31.260753 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.260383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54272)
0809 22:59:31.260531 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.260611 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 22:59:31.260939 32302 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [term 2 FOLLOWER]: Refusing update from remote peer bae2d3a95a0f4da695a51f5be350262f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 22:59:31.261904 693 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 22:59:31.262125 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.261687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54288)
0809 22:59:31.261870 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.261963 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:59:31.262358 32496 consensus_peers.cc:487] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f -> Peer 66318d8fd4694039bb253dc9c4f3640e (127.24.31.254:46425): Couldn't send request to peer 66318d8fd4694039bb253dc9c4f3640e. Status: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 22:59:31.264690 687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bae2d3a95a0f4da695a51f5be350262f. Latest consensus state: current_term: 2 leader_uuid: "bae2d3a95a0f4da695a51f5be350262f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:31.265010 687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:59:31.266436 683 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "bae2d3a95a0f4da695a51f5be350262f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:31.266836 683 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:31.267488 687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "bae2d3a95a0f4da695a51f5be350262f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "66318d8fd4694039bb253dc9c4f3640e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46425 } } peers { permanent_uuid: "93cb005232ed48a78e2d5b8898761e45" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 38021 } attrs { promote: false } } peers { permanent_uuid: "bae2d3a95a0f4da695a51f5be350262f" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 33965 } attrs { promote: false } } }
I20260809 22:59:31.267777 687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93cb005232ed48a78e2d5b8898761e45 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 22:59:31.269953 699 mvcc.cc:205] Tried to move back new op lower bound from 7316751856678174720 to 7316751856433799168. Current Snapshot: MvccSnapshot[applied={T|T < 7316751856678174720 or (T in {7316751856678174720})}]
I20260809 22:59:31.287329 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.275878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:45727 (local address 127.0.0.1:52118)
0809 22:59:31.276140 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.276162 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:31.276311 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:31.278970 (+ 2659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:31.278978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:31.279014 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:31.279424 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.279439 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.281656 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.281665 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.284465 (+ 2800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.284486 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.286544 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.286550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.286750 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.286806 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 22:59:31.286877 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 22:59:31.287017 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 22:59:31.288810 706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.276349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.24.31.194:45727)
0809 22:59:31.278528 (+ 2179us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.278537 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.278563 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:59:31.278656 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.278664 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.278761 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.279601 (+ 840us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:31.279826 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.281438 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.284842 (+ 3404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.286271 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.288240 (+ 1969us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.288341 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.288392 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.288524 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2011,"thread_start_us":4721,"threads_started":1}
I20260809 22:59:31.289678 358 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed51501de9c4da8a4b950ff7481b9ec (DEFAULT_TABLE table=test-table [id=9574482a8e594cfa82869b6b29d3b487]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:31.290191 358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed51501de9c4da8a4b950ff7481b9ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:31.292057 686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.275298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33374)
0809 22:59:31.275511 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.275534 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:31.275653 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:31.286281 (+ 10628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:31.286286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:31.286319 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:31.286557 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.286570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.287938 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.287942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.289752 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.289767 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.291295 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.291305 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.291530 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.291563 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 22:59:31.291629 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 22:59:31.291769 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 22:59:31.292929 705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.275421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.24.31.193:36033)
0809 22:59:31.285866 (+ 10445us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.285875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.285902 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:59:31.285999 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.286007 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.286107 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.286231 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:31.286742 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.287770 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.289948 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.291094 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.292324 (+ 1230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.292426 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.292477 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.292607 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10288,"thread_start_us":10143,"threads_started":1}
I20260809 22:59:31.293705 32647 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed51501de9c4da8a4b950ff7481b9ec (DEFAULT_TABLE table=test-table [id=9574482a8e594cfa82869b6b29d3b487]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:31.294137 32647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed51501de9c4da8a4b950ff7481b9ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:31.295274 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.278091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:43257 (local address 127.0.0.1:58452)
0809 22:59:31.287540 (+ 9449us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.287557 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:31.287650 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:31.288078 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:31.288086 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:31.288126 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:31.288420 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.288435 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.290667 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.290676 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.292358 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.292372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.294518 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.294525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.294711 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.294745 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 22:59:31.294811 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 22:59:31.294956 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9364,"mutex_wait_us":1,"thread_start_us":13134,"threads_started":1}
I20260809 22:59:31.296077 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.278650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.24.31.195:43257)
0809 22:59:31.283096 (+ 4446us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.283105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.287627 (+ 4522us) server_negotiation.cc:407] Connection header received
0809 22:59:31.287762 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.287770 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.287872 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.288585 (+ 713us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:31.288804 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.290475 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.292562 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.293854 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.295498 (+ 1644us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.295623 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.295672 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.295804 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4276,"thread_start_us":3980,"threads_started":1}
I20260809 22:59:31.299194 709 tablet_bootstrap.cc:492] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: Bootstrap starting.
I20260809 22:59:31.299124 524 tablet_service.cc:1467] Processing CreateTablet for tablet 7ed51501de9c4da8a4b950ff7481b9ec (DEFAULT_TABLE table=test-table [id=9574482a8e594cfa82869b6b29d3b487]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:31.299579 524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ed51501de9c4da8a4b950ff7481b9ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:31.301137 709 tablet_bootstrap.cc:654] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:31.301383 710 tablet_bootstrap.cc:492] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356: Bootstrap starting.
I20260809 22:59:31.303051 710 tablet_bootstrap.cc:654] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:31.303570 709 tablet_bootstrap.cc:492] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: No bootstrap required, opened a new log
I20260809 22:59:31.303733 709 ts_tablet_manager.cc:1397] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 22:59:31.304531 709 raft_consensus.cc:348] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:31.304781 709 raft_consensus.cc:374] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:31.304917 709 raft_consensus.cc:729] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6df5178167940189db0b8836ca61bbe, State: Initialized, Role: FOLLOWER
I20260809 22:59:31.305347 710 tablet_bootstrap.cc:492] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356: No bootstrap required, opened a new log
I20260809 22:59:31.305236 709 consensus_queue.cc:260] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:31.305555 710 ts_tablet_manager.cc:1397] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.003s
I20260809 22:59:31.306176 709 ts_tablet_manager.cc:1428] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 22:59:31.306465 710 raft_consensus.cc:348] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:31.306824 710 raft_consensus.cc:374] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:31.306980 710 raft_consensus.cc:729] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68596e5ddb484442af4a90aaa1af1356, State: Initialized, Role: FOLLOWER
I20260809 22:59:31.307327 710 consensus_queue.cc:260] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:31.310251 432 heartbeater.cc:503] Master 127.24.31.252:33965 was elected leader, sending a full tablet report...
I20260809 22:59:31.315718 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.306510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.194:40311)
0809 22:59:31.315130 (+ 8620us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.315327 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8484,"thread_start_us":8315,"threads_started":1}
I20260809 22:59:31.317135 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.316646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.194:36251)
0809 22:59:31.316851 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.316961 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 22:59:31.317566 715 tablet_bootstrap.cc:492] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: Bootstrap starting.
I20260809 22:59:31.319310 715 tablet_bootstrap.cc:654] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:31.321192 710 ts_tablet_manager.cc:1428] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20260809 22:59:31.322726 715 tablet_bootstrap.cc:492] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: No bootstrap required, opened a new log
I20260809 22:59:31.322948 715 ts_tablet_manager.cc:1397] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 22:59:31.323395 32727 heartbeater.cc:503] Master 127.24.31.252:33965 was elected leader, sending a full tablet report...
I20260809 22:59:31.323968 715 raft_consensus.cc:348] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:31.324245 715 raft_consensus.cc:374] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:31.324376 715 raft_consensus.cc:729] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d25cc28128cc4c5093710787c012491a, State: Initialized, Role: FOLLOWER
I20260809 22:59:31.324680 715 consensus_queue.cc:260] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:31.325906 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.321403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.193:49875)
0809 22:59:31.325506 (+ 4103us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.325656 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3933,"threads_started":1}
I20260809 22:59:31.326692 715 ts_tablet_manager.cc:1428] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260809 22:59:31.327286 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.326723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.193:44155)
0809 22:59:31.327067 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.327152 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234}
I20260809 22:59:31.328763 600 heartbeater.cc:503] Master 127.24.31.252:33965 was elected leader, sending a full tablet report...
I20260809 22:59:31.328857 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.326872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.195:45705)
0809 22:59:31.328333 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.328534 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1341,"thread_start_us":1176,"threads_started":1}
I20260809 22:59:31.330417 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.329681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.195:43003)
0809 22:59:31.330179 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.330269 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 22:59:31.369879 713 raft_consensus.cc:482] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:31.370157 713 raft_consensus.cc:504] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:31.371757 713 leader_election.cc:288] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033), d25cc28128cc4c5093710787c012491a (127.24.31.195:43257)
I20260809 22:59:31.378026 705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.372014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:35095 (local address 127.24.31.193:36033)
0809 22:59:31.372260 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.372279 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.372339 (+ 60us) server_negotiation.cc:407] Connection header received
0809 22:59:31.372614 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.372619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.372747 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.372827 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:31.373459 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.374461 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.376325 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.377085 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.377285 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.377363 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.377636 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.377779 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:59:31.378067 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.371936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.24.31.194:35095)
0809 22:59:31.372226 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.372263 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:31.372441 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:31.372887 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:31.372892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:31.372954 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:31.373309 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.373321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.374638 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.374643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.376159 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.376175 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.377291 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.377297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.377465 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.377489 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:59:31.377609 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 22:59:31.377759 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":1,"spinlock_wait_cycles":384}
I20260809 22:59:31.378492 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.372189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:43257 (local address 127.24.31.194:41921)
0809 22:59:31.372791 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.372826 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:31.372959 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:31.373323 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:31.373327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:31.373352 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:31.373659 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.373667 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.374996 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.375000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.376410 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:31.376423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.377640 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.377647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:31.377818 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.377841 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 22:59:31.377942 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 22:59:31.378068 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":41,"spinlock_wait_cycles":10496,"thread_start_us":361,"threads_started":1}
I20260809 22:59:31.378508 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.372200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:41921 (local address 127.24.31.195:43257)
0809 22:59:31.372387 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.372393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.372926 (+ 533us) server_negotiation.cc:407] Connection header received
0809 22:59:31.373101 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.373105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.373208 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.373290 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:31.373839 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.374874 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.376582 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.377412 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.377595 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.377679 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.377979 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.378111 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:59:31.378880 32671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed51501de9c4da8a4b950ff7481b9ec" candidate_uuid: "a6df5178167940189db0b8836ca61bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68596e5ddb484442af4a90aaa1af1356" is_pre_election: true
I20260809 22:59:31.379084 32671 raft_consensus.cc:2454] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6df5178167940189db0b8836ca61bbe in term 0.
I20260809 22:59:31.379534 301 leader_election.cc:302] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68596e5ddb484442af4a90aaa1af1356, a6df5178167940189db0b8836ca61bbe; no voters:
I20260809 22:59:31.379930 713 raft_consensus.cc:2781] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:31.380117 713 raft_consensus.cc:482] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:31.380226 713 raft_consensus.cc:3037] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:31.380066 547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed51501de9c4da8a4b950ff7481b9ec" candidate_uuid: "a6df5178167940189db0b8836ca61bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25cc28128cc4c5093710787c012491a" is_pre_election: true
I20260809 22:59:31.380496 547 raft_consensus.cc:2454] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6df5178167940189db0b8836ca61bbe in term 0.
I20260809 22:59:31.383270 713 raft_consensus.cc:504] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:31.383991 713 leader_election.cc:288] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [CANDIDATE]: Term 1 election: Requested vote from peers 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033), d25cc28128cc4c5093710787c012491a (127.24.31.195:43257)
I20260809 22:59:31.384477 32671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed51501de9c4da8a4b950ff7481b9ec" candidate_uuid: "a6df5178167940189db0b8836ca61bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68596e5ddb484442af4a90aaa1af1356"
I20260809 22:59:31.384713 32671 raft_consensus.cc:3037] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:31.384684 547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ed51501de9c4da8a4b950ff7481b9ec" candidate_uuid: "a6df5178167940189db0b8836ca61bbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d25cc28128cc4c5093710787c012491a"
I20260809 22:59:31.384943 547 raft_consensus.cc:3037] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:31.387388 32671 raft_consensus.cc:2454] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6df5178167940189db0b8836ca61bbe in term 1.
I20260809 22:59:31.388315 547 raft_consensus.cc:2454] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6df5178167940189db0b8836ca61bbe in term 1.
I20260809 22:59:31.388453 301 leader_election.cc:302] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68596e5ddb484442af4a90aaa1af1356, a6df5178167940189db0b8836ca61bbe; no voters:
I20260809 22:59:31.388792 713 raft_consensus.cc:2781] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:31.389389 713 raft_consensus.cc:686] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 1 LEADER]: Becoming Leader. State: Replica: a6df5178167940189db0b8836ca61bbe, State: Running, Role: LEADER
I20260809 22:59:31.389837 713 consensus_queue.cc:237] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } }
I20260809 22:59:31.390822 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.390345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.194:38889)
0809 22:59:31.390502 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.390638 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 22:59:31.393628 32512 catalog_manager.cc:5495] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe reported cstate change: term changed from 0 to 1, leader changed from <none> to a6df5178167940189db0b8836ca61bbe (127.24.31.194). New cstate: current_term: 1 leader_uuid: "a6df5178167940189db0b8836ca61bbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6df5178167940189db0b8836ca61bbe" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 45727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 } health_report { overall_health: UNKNOWN } } }
I20260809 22:59:31.411890 706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.407177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.24.31.194:45727)
0809 22:59:31.407368 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.407373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.407402 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:59:31.407484 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.407488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.407562 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.407689 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:31.407988 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.408689 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.410006 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.410448 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.410539 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.410894 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:31.411433 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:31.411549 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.411621 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.411683 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:59:31.415930 32671 raft_consensus.cc:1261] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [term 1 FOLLOWER]: Refusing update from remote peer a6df5178167940189db0b8836ca61bbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:31.416205 547 raft_consensus.cc:1261] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [term 1 FOLLOWER]: Refusing update from remote peer a6df5178167940189db0b8836ca61bbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:31.416705 723 consensus_queue.cc:1035] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "68596e5ddb484442af4a90aaa1af1356" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 36033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:31.417086 713 consensus_queue.cc:1035] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d25cc28128cc4c5093710787c012491a" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 43257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:31.420200 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.419503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.195:43259)
0809 22:59:31.419792 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.419949 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 22:59:31.420261 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.419162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.193:40025)
0809 22:59:31.419857 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.419979 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531}
W20260809 22:59:31.428213 435 tablet.cc:2369] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:59:31.430807 721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.430253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.195:46169)
0809 22:59:31.430552 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.430644 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260809 22:59:31.432144 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.431671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.194:52843)
0809 22:59:31.431920 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.432007 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260809 22:59:31.439881 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.439343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.194:59981)
0809 22:59:31.439576 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.439684 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260809 22:59:31.440101 728 mvcc.cc:205] Tried to move back new op lower bound from 7316751857313005568 to 7316751857217728512. Current Snapshot: MvccSnapshot[applied={T|T < 7316751857313005568}]
I20260809 22:59:31.441228 719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.440806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.24.31.193:54889)
0809 22:59:31.440959 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.441047 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 22:59:31.447935 32512 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:31.448249 32512 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:31.454320 32512 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bae2d3a95a0f4da695a51f5be350262f: Sending DeleteTablet for 3 replicas of tablet 7ed51501de9c4da8a4b950ff7481b9ec
I20260809 22:59:31.455348 358 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ed51501de9c4da8a4b950ff7481b9ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:31 UTC) from {username='oryx', principal='oryx/127.24.31.252@KRBTEST.COM'} at 127.0.0.1:52118
I20260809 22:59:31.455403 32647 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ed51501de9c4da8a4b950ff7481b9ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:31 UTC) from {username='oryx', principal='oryx/127.24.31.252@KRBTEST.COM'} at 127.0.0.1:33374
I20260809 22:59:31.456076 524 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ed51501de9c4da8a4b950ff7481b9ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:31 UTC) from {username='oryx', principal='oryx/127.24.31.252@KRBTEST.COM'} at 127.0.0.1:58452
I20260809 22:59:31.456967 739 tablet_replica.cc:330] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: stopping tablet replica
I20260809 22:59:31.457319 739 raft_consensus.cc:2229] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:31.457834 739 raft_consensus.cc:2258] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:31.457906 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 32486
I20260809 22:59:31.464026 739 ts_tablet_manager.cc:1905] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:31.476446 739 ts_tablet_manager.cc:1918] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:31.476593 739 log.cc:1196] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/7ed51501de9c4da8a4b950ff7481b9ec
I20260809 22:59:31.477097 739 ts_tablet_manager.cc:1939] T 7ed51501de9c4da8a4b950ff7481b9ec P d25cc28128cc4c5093710787c012491a: Deleting consensus metadata
I20260809 22:59:31.478515 738 tablet_replica.cc:330] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356: stopping tablet replica
I20260809 22:59:31.478734 738 raft_consensus.cc:2229] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:31.479080 738 raft_consensus.cc:2258] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:31.479700 737 tablet_replica.cc:330] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: stopping tablet replica
I20260809 22:59:31.479902 737 raft_consensus.cc:2229] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:59:31.480597 737 raft_consensus.cc:2258] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:31.482183 737 ts_tablet_manager.cc:1905] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:31.485780 738 ts_tablet_manager.cc:1905] T 7ed51501de9c4da8a4b950ff7481b9ec P 68596e5ddb484442af4a90aaa1af1356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:31.487887 32496 catalog_manager.cc:4841] TS d25cc28128cc4c5093710787c012491a (127.24.31.195:43257): tablet 7ed51501de9c4da8a4b950ff7481b9ec (table test-table [id=9574482a8e594cfa82869b6b29d3b487]) successfully deleted
I20260809 22:59:31.489334 737 ts_tablet_manager.cc:1918] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:31.489490 737 log.cc:1196] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/7ed51501de9c4da8a4b950ff7481b9ec
I20260809 22:59:31.490036 737 ts_tablet_manager.cc:1939] T 7ed51501de9c4da8a4b950ff7481b9ec P a6df5178167940189db0b8836ca61bbe: Deleting consensus metadata
I20260809 22:59:31.491230 32495 catalog_manager.cc:4841] TS a6df5178167940189db0b8836ca61bbe (127.24.31.194:45727): tablet 7ed51501de9c4da8a4b950ff7481b9ec (table test-table [id=9574482a8e594cfa82869b6b29d3b487]) successfully deleted
I20260809 22:59:31.492591 705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.458944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.24.31.193:36033)
0809 22:59:31.459159 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.459169 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.473876 (+ 14707us) server_negotiation.cc:407] Connection header received
0809 22:59:31.474052 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.474062 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.474206 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.474384 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:31.476824 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.478594 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.480415 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.481472 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.481649 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.484617 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:31.484640 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:31.484668 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:31.484726 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:31.489134 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:31.490002 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:31.490014 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:31.490021 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:31.490262 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:31.490662 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:31.490671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:31.490675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:31.491216 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:31.491606 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.491903 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.492148 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 22:59:31.493360 32720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:31.591475 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 32486
I20260809 22:59:31.611996 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.611457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33390)
0809 22:59:31.611687 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.611797 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 22:59:31.612112 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 32741
W20260809 22:59:31.612748 32498 catalog_manager.cc:4584] TS 68596e5ddb484442af4a90aaa1af1356 (127.24.31.193:36033): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ed51501de9c4da8a4b950ff7481b9ec: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
I20260809 22:59:31.634616 706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.612846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52130 (local address 127.24.31.194:45727)
0809 22:59:31.613025 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.613030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.613387 (+ 357us) server_negotiation.cc:407] Connection header received
0809 22:59:31.613569 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.613576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.613691 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.613797 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:31.615622 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.616885 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.618974 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.619702 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.619823 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.627667 (+ 7844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:31.627705 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:31.627729 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:31.627781 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:31.631467 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:31.632407 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:31.632417 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:31.632425 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:31.632617 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:31.633091 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:31.633094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:31.633096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:31.633514 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:31.633752 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.634097 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.634266 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 22:59:31.635388 427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:31.673475 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.672967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33400)
0809 22:59:31.673219 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.673307 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 22:59:31.674578 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.674164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33410)
0809 22:59:31.674333 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.674440 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 22:59:31.743042 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.742600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33424)
0809 22:59:31.742801 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.742887 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 22:59:31.744046 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.743699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33438)
0809 22:59:31.743856 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.743931 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 22:59:31.756541 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 32741
I20260809 22:59:31.779291 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 445
I20260809 22:59:31.793504 708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.780001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58468 (local address 127.24.31.195:43257)
0809 22:59:31.780195 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:31.780203 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:31.780262 (+ 59us) server_negotiation.cc:407] Connection header received
0809 22:59:31.780517 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:31.780521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:31.780586 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:31.780700 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:31.782192 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.782997 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.784239 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:31.784624 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:31.784754 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:31.787522 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:31.787538 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:31.787555 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:31.787589 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:31.790676 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:31.791358 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:31.791370 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:31.791374 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:31.791556 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:31.791929 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:31.791935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:31.791938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:31.792305 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:31.792618 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:31.792914 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 22:59:31.793175 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 22:59:31.794162 593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:31.845822 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.845350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33444)
0809 22:59:31.845572 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.845658 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 22:59:31.846850 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.846508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33450)
0809 22:59:31.846645 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.846719 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 22:59:31.866562 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.866170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54296)
0809 22:59:31.866339 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.866416 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 22:59:31.867581 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:31.867206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46425 (local address 127.0.0.1:54304)
0809 22:59:31.867367 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:31.867462 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 22:59:31.918809 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 445
I20260809 22:59:31.946437 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 32128
I20260809 22:59:31.947791 32323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:32.018894 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:32.018366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33456)
0809 22:59:32.018619 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:32.018728 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260809 22:59:32.019945 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:32.019613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:36033 (local address 127.0.0.1:33460)
0809 22:59:32.019738 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:32.019828 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:36033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 22:59:32.075822 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 32128
I20260809 22:59:32.099910 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 32239
I20260809 22:59:32.101019 32546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:32.223930 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 32239
2026-08-09T22:59:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9788 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:59:32 dist-test-slave-vxj2 krb5kdc[750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:59:32 dist-test-slave-vxj2 krb5kdc[750](info): set up 2 sockets
Aug 09 22:59:32 dist-test-slave-vxj2 krb5kdc[750](info): commencing operation
krb5kdc: starting...
W20260809 22:59:33.431576 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:59:33 dist-test-slave-vxj2 krb5kdc[750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:59:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.254.keytab.
Entry for principal oryx/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.254.keytab.
I20260809 22:59:33.680985 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44915
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.254.keytab
--principal=oryx/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:42915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:44915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:34.009699 766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:34.010053 766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:34.010218 766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:34.017592 766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:34.017726 766 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 22:59:34.017823 766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:34.017938 766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:34.018025 766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:34.018115 766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:34.028208 766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:44915
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.254.keytab
--principal=oryx/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:34.029557 766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:34.030881 766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:34.043035 772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:34.043041 776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:34.043068 773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:34.044672 766 server_base.cc:1034] running on GCE node
I20260809 22:59:34.045584 766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:34.047109 766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:34.048367 766 hybrid_clock.cc:648] HybridClock initialized: now 1786316374048260 us; error 53 us; skew 500 ppm
I20260809 22:59:34.048734 766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:34 dist-test-slave-vxj2 krb5kdc[750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316374, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:34.052399 766 init.cc:377] Logged in from keytab as oryx/127.24.31.254@KRBTEST.COM (short username oryx)
I20260809 22:59:34.054176 766 webserver.cc:460] Webserver started at http://127.24.31.254:40581/ using document root <none> and password file <none>
I20260809 22:59:34.054951 766 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:34.055047 766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:34.055418 766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:34.058688 766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "3cb84b66bbf245e3a50b36ab42b2035e"
format_stamp: "Formatted at 2026-08-09 22:59:34 on dist-test-slave-vxj2"
server_key: "708c2d4ef9115a1fe35034b6fbccb442"
server_key_iv: "2c3a0a8c7abbb2fcebde8fb3b1cb68a3"
server_key_version: "encryptionkey@0"
I20260809 22:59:34.059566 766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "3cb84b66bbf245e3a50b36ab42b2035e"
format_stamp: "Formatted at 2026-08-09 22:59:34 on dist-test-slave-vxj2"
server_key: "708c2d4ef9115a1fe35034b6fbccb442"
server_key_iv: "2c3a0a8c7abbb2fcebde8fb3b1cb68a3"
server_key_version: "encryptionkey@0"
I20260809 22:59:34.064999 766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:59:34.068866 787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:34.070292 766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:34.070483 766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "3cb84b66bbf245e3a50b36ab42b2035e"
format_stamp: "Formatted at 2026-08-09 22:59:34 on dist-test-slave-vxj2"
server_key: "708c2d4ef9115a1fe35034b6fbccb442"
server_key_iv: "2c3a0a8c7abbb2fcebde8fb3b1cb68a3"
server_key_version: "encryptionkey@0"
I20260809 22:59:34.070696 766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:34.095582 766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:34.099910 766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:34.100215 766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:34.124339 766 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:44915
I20260809 22:59:34.124377 853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:44915 every 8 connection(s)
I20260809 22:59:34.125838 766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:34.129637 854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:34.134802 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 766
I20260809 22:59:34.135043 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:34.135854 24703 external_mini_cluster.cc:1428] Setting key 5aa60764d33b7035c97a1e9cd1e69e68
I20260809 22:59:34.140894 854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e: Bootstrap starting.
I20260809 22:59:34.144600 854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:34.145862 854 log.cc:824] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:34.149161 854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e: No bootstrap required, opened a new log
Aug 09 22:59:34 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:34.159685 854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb84b66bbf245e3a50b36ab42b2035e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44915 } }
I20260809 22:59:34.160148 854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:34.160246 854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb84b66bbf245e3a50b36ab42b2035e, State: Initialized, Role: FOLLOWER
I20260809 22:59:34.160924 854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb84b66bbf245e3a50b36ab42b2035e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44915 } }
I20260809 22:59:34.161237 854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:34.161439 857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:34.137992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60162 (local address 127.24.31.254:44915)
0809 22:59:34.138836 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:34.138848 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:34.138891 (+ 43us) server_negotiation.cc:407] Connection header received
0809 22:59:34.140537 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:34.140561 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:34.140913 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:34.141319 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:34.143182 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.144453 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.146266 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.147374 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.147508 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:34.152178 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:34.152188 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:34.152211 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:34.152258 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:34.156319 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:34.157309 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:34.157318 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:34.157326 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:34.157566 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:34.158128 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:34.158135 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:34.158141 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:34.158883 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:34.159181 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:34.159573 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 22:59:34.160017 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":409,"threads_started":1}
I20260809 22:59:34.161397 854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:34.161659 854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:34.164383 854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb84b66bbf245e3a50b36ab42b2035e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44915 } }
I20260809 22:59:34.164829 854 leader_election.cc:302] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cb84b66bbf245e3a50b36ab42b2035e; no voters:
I20260809 22:59:34.165372 854 leader_election.cc:288] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:34.165521 859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:34.166231 859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [term 1 LEADER]: Becoming Leader. State: Replica: 3cb84b66bbf245e3a50b36ab42b2035e, State: Running, Role: LEADER
I20260809 22:59:34.166767 859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb84b66bbf245e3a50b36ab42b2035e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44915 } }
I20260809 22:59:34.167411 854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:34.169625 860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cb84b66bbf245e3a50b36ab42b2035e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb84b66bbf245e3a50b36ab42b2035e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44915 } } }
I20260809 22:59:34.170077 860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:34.170909 861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cb84b66bbf245e3a50b36ab42b2035e. Latest consensus state: current_term: 1 leader_uuid: "3cb84b66bbf245e3a50b36ab42b2035e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cb84b66bbf245e3a50b36ab42b2035e" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44915 } } }
I20260809 22:59:34.171303 861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:34.172235 864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:34.177714 864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:34.178393 877 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 22:59:34.186373 864 catalog_manager.cc:1347] Generated new cluster ID: 9d4008fce4714bd2a2d6d1656b0fb344
I20260809 22:59:34.186501 864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:34.213104 864 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:34.214939 864 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:34.225368 864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e: Generated new TSK 0
I20260809 22:59:34.226359 864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab.
Entry for principal oryx/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab.
I20260809 22:59:34.300482 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44915
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab
--principal=oryx/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:42915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:34.648953 882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:34.649289 882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:34.649425 882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:34.656773 882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:34.656876 882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:34.656965 882 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:59:34.657044 882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:34.666685 882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab
--principal=oryx/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.31.254:44915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:34.667904 882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:34.669327 882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:34.682745 888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:34.682874 887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:34.682869 891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:34.683868 882 server_base.cc:1034] running on GCE node
I20260809 22:59:34.684506 882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:34.685465 882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:34.686676 882 hybrid_clock.cc:648] HybridClock initialized: now 1786316374686626 us; error 29 us; skew 500 ppm
I20260809 22:59:34.686968 882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:34 dist-test-slave-vxj2 krb5kdc[750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316374, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:34.690086 882 init.cc:377] Logged in from keytab as oryx/127.24.31.193@KRBTEST.COM (short username oryx)
I20260809 22:59:34.691432 882 webserver.cc:460] Webserver started at http://127.24.31.193:35223/ using document root <none> and password file <none>
I20260809 22:59:34.692106 882 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:34.692235 882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:34.692551 882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:34.695396 882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "85d3f03e4888498db89cfd644bcf693d"
format_stamp: "Formatted at 2026-08-09 22:59:34 on dist-test-slave-vxj2"
server_key: "b447585aec6cb3179fcd0fbb149697e3"
server_key_iv: "8d37bdc49e72f360dc069c0d4b291956"
server_key_version: "encryptionkey@0"
I20260809 22:59:34.696192 882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "85d3f03e4888498db89cfd644bcf693d"
format_stamp: "Formatted at 2026-08-09 22:59:34 on dist-test-slave-vxj2"
server_key: "b447585aec6cb3179fcd0fbb149697e3"
server_key_iv: "8d37bdc49e72f360dc069c0d4b291956"
server_key_version: "encryptionkey@0"
I20260809 22:59:34.701463 882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:59:34.704855 901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:34.706198 882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:34.706369 882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "85d3f03e4888498db89cfd644bcf693d"
format_stamp: "Formatted at 2026-08-09 22:59:34 on dist-test-slave-vxj2"
server_key: "b447585aec6cb3179fcd0fbb149697e3"
server_key_iv: "8d37bdc49e72f360dc069c0d4b291956"
server_key_version: "encryptionkey@0"
I20260809 22:59:34.706607 882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:34.735306 882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:34.739485 882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:34.739768 882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:34.742532 882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:34.742642 882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:34.742851 882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:34.742990 882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 09 22:59:34 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316374, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:34.767661 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:34.743042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:55065 (local address 127.24.31.254:44915)
0809 22:59:34.743543 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:34.743549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:34.744820 (+ 1271us) server_negotiation.cc:407] Connection header received
0809 22:59:34.747126 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:34.747132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:34.747224 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:34.747347 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:34.749703 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.751058 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.753620 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.754385 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.754579 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:34.759635 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:34.759643 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:34.759648 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:34.759697 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:34.763717 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:34.764651 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:34.764659 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:34.764666 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:34.764846 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:34.765390 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:34.765396 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:34.765399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:34.765690 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:34.765873 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:34.767085 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0809 22:59:34.767345 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":263,"threads_started":1}
I20260809 22:59:34.769747 912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:34.743731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44915 (local address 127.24.31.193:55065)
0809 22:59:34.744630 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:34.744689 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:34.746848 (+ 2159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:34.747611 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:34.747625 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:34.748442 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:34.749415 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:34.749439 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.751252 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.751259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:34.753472 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:34.753484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.754679 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.754691 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:34.754947 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:34.755773 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:34.755807 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:34.759363 (+ 3556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:34.763866 (+ 4503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:34.763882 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:34.763904 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:34.764493 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:34.764984 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:34.764989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:34.764992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:34.765260 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:34.765867 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:34.765875 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:34.766502 (+ 627us) client_negotiation.cc:769] Sending connection context
0809 22:59:34.766901 (+ 399us) client_negotiation.cc:240] Negotiation successful
0809 22:59:34.767704 (+ 803us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":391,"threads_started":1}
I20260809 22:59:34.790458 997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:34.776226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44915 (local address 127.24.31.193:37921)
0809 22:59:34.776973 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:34.776999 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:34.777198 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:34.777707 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:34.777710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:34.778182 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:34.778730 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:34.778741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.780058 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.780063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:34.781477 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:34.781489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.782359 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.782366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:34.782530 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:34.783236 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:34.783261 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:34.783982 (+ 721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:34.787412 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:34.787417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:34.787421 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:34.788007 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:34.788558 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:34.788563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:34.788566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:34.788780 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:34.789592 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:34.789597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:34.789758 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 22:59:34.789878 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 22:59:34.790114 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":380,"threads_started":1}
I20260809 22:59:34.790887 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:34.776226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:37921 (local address 127.24.31.254:44915)
0809 22:59:34.776450 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:34.776460 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:34.777098 (+ 638us) server_negotiation.cc:407] Connection header received
0809 22:59:34.777370 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:34.777375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:34.777468 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:34.777565 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:34.778869 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.779921 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.781623 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:34.782165 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:34.782296 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:34.784166 (+ 1870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:34.784177 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:34.784180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:34.784211 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:34.787284 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:34.788237 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:34.788244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:34.788249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:34.788434 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:34.788929 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:34.788937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:34.788943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:34.789426 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:34.789609 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:34.790043 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 22:59:34.790383 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 22:59:34.794214 882 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:42483
I20260809 22:59:34.794251 1037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:42483 every 8 connection(s)
I20260809 22:59:34.795429 882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:34.797396 1038 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44915
I20260809 22:59:34.797621 1038 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:34.798251 1038 heartbeater.cc:511] Master 127.24.31.254:44915 requested a full tablet report, sending...
I20260809 22:59:34.800192 806 ts_manager.cc:194] Registered new tserver with Master: 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483)
I20260809 22:59:34.801693 806 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:55065
I20260809 22:59:34.804600 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 882
I20260809 22:59:34.804761 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:59:34.805469 24703 external_mini_cluster.cc:1428] Setting key 9e6d7270c646993db5e725913ebcbdc9
WARNING: no policy specified for oryx/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab.
Entry for principal oryx/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab.
I20260809 22:59:34.856750 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44915
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab
--principal=oryx/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:42915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:35.187510 1042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:35.187786 1042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:35.187995 1042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:35.195135 1042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:35.195273 1042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:35.195500 1042 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:59:35.195712 1042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:35.207019 1042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab
--principal=oryx/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.31.254:44915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:35.208428 1042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:35.210083 1042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:35.223364 1048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:35.223476 1055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:35.223486 1049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:35.224421 1042 server_base.cc:1034] running on GCE node
I20260809 22:59:35.224969 1042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:35.225929 1042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:35.227147 1042 hybrid_clock.cc:648] HybridClock initialized: now 1786316375227092 us; error 33 us; skew 500 ppm
I20260809 22:59:35.227449 1042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:35 dist-test-slave-vxj2 krb5kdc[750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316375, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:35.230583 1042 init.cc:377] Logged in from keytab as oryx/127.24.31.194@KRBTEST.COM (short username oryx)
I20260809 22:59:35.231971 1042 webserver.cc:460] Webserver started at http://127.24.31.194:43217/ using document root <none> and password file <none>
I20260809 22:59:35.232669 1042 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:35.232777 1042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:35.233094 1042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:35.235970 1042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "87140d10272047d5a56cc9b5e6f28aad"
format_stamp: "Formatted at 2026-08-09 22:59:35 on dist-test-slave-vxj2"
server_key: "1fe92dda1c03452b2587f3a36255f961"
server_key_iv: "f2b0968a09e76ee70d95833d5f3b6f72"
server_key_version: "encryptionkey@0"
I20260809 22:59:35.236830 1042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "87140d10272047d5a56cc9b5e6f28aad"
format_stamp: "Formatted at 2026-08-09 22:59:35 on dist-test-slave-vxj2"
server_key: "1fe92dda1c03452b2587f3a36255f961"
server_key_iv: "f2b0968a09e76ee70d95833d5f3b6f72"
server_key_version: "encryptionkey@0"
I20260809 22:59:35.242012 1042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 22:59:35.245460 1062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:35.246799 1042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:59:35.246950 1042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "87140d10272047d5a56cc9b5e6f28aad"
format_stamp: "Formatted at 2026-08-09 22:59:35 on dist-test-slave-vxj2"
server_key: "1fe92dda1c03452b2587f3a36255f961"
server_key_iv: "f2b0968a09e76ee70d95833d5f3b6f72"
server_key_version: "encryptionkey@0"
I20260809 22:59:35.247176 1042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:35.266615 1042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:35.270682 1042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:35.270952 1042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:35.274151 1042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:35.274276 1042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:35.274420 1042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:35.274516 1042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:59:35 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316375, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:35.300057 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:35.274440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:37437 (local address 127.24.31.254:44915)
0809 22:59:35.274641 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:35.274647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:35.276338 (+ 1691us) server_negotiation.cc:407] Connection header received
0809 22:59:35.279325 (+ 2987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:35.279330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:35.279399 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:35.279486 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:35.281928 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.283124 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.285822 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.286463 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.286610 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:35.291719 (+ 5109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:35.291725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:35.291731 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:35.291778 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:35.295788 (+ 4010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:35.297089 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:35.297098 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:35.297104 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:35.297301 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:35.297919 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:35.297925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:35.297928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:35.298343 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:35.298517 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:35.299479 (+ 962us) server_negotiation.cc:299] Negotiation successful
0809 22:59:35.299699 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:59:35.301515 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:35.275005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44915 (local address 127.24.31.194:37437)
0809 22:59:35.276116 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:35.276186 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:35.279004 (+ 2818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:35.279772 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:35.279787 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:35.280600 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:35.281682 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:35.281719 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.283384 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.283388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:35.285663 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:35.285675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.286732 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.286739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:35.286970 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:35.287671 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:35.287717 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:35.291446 (+ 3729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:35.296013 (+ 4567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:35.296025 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:35.296044 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:35.296883 (+ 839us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:35.297459 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:35.297466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:35.297471 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:35.297788 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:35.298594 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:35.298607 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:35.299089 (+ 482us) client_negotiation.cc:769] Sending connection context
0809 22:59:35.299354 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 22:59:35.299928 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":415,"threads_started":1}
I20260809 22:59:35.321975 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:35.307997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:43245 (local address 127.24.31.254:44915)
0809 22:59:35.308242 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:35.308250 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:35.308717 (+ 467us) server_negotiation.cc:407] Connection header received
0809 22:59:35.308902 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:35.308907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:35.308977 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:35.309075 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:35.310439 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.311674 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.313375 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.313804 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.313966 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:35.315811 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:35.315817 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:35.315820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:35.315849 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:35.318696 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:35.319560 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:35.319565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:35.319571 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:35.319706 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:35.320169 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:35.320179 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:35.320185 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:35.320815 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:35.321036 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:35.321428 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 22:59:35.321640 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:59:35.322304 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:35.307960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44915 (local address 127.24.31.194:43245)
0809 22:59:35.308622 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:35.308637 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:35.308785 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:35.309095 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:35.309100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:35.309660 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:35.310240 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:35.310258 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.311850 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.311854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:35.313182 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:35.313202 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.314013 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.314018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:35.314191 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:35.314902 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:35.314918 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:35.315621 (+ 703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:35.318867 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:35.318872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:35.318884 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:35.319380 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:35.319822 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:35.319827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:35.319830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:35.320014 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:35.320962 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:35.320966 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:35.321122 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 22:59:35.321337 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 22:59:35.321687 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":324,"threads_started":1}
I20260809 22:59:35.326366 1042 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:34911
I20260809 22:59:35.326397 1195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:34911 every 8 connection(s)
I20260809 22:59:35.327702 1042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:35.329460 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 1042
I20260809 22:59:35.329658 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:59:35.330169 24703 external_mini_cluster.cc:1428] Setting key 35c307f036296f010fadd989487fd34b
I20260809 22:59:35.329840 1196 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44915
I20260809 22:59:35.331249 1196 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:35.332022 1196 heartbeater.cc:511] Master 127.24.31.254:44915 requested a full tablet report, sending...
I20260809 22:59:35.333552 806 ts_manager.cc:194] Registered new tserver with Master: 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194:34911)
I20260809 22:59:35.334503 806 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:37437
WARNING: no policy specified for oryx/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab.
Entry for principal oryx/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab.
I20260809 22:59:35.385382 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44915
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab
--principal=oryx/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:42915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:35.722537 1201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:35.722823 1201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:35.722923 1201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:35.730119 1201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:35.730222 1201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:35.730309 1201 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 22:59:35.730391 1201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:35.740033 1201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab
--principal=oryx/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.31.254:44915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:35.741250 1201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:35.742492 1201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:35.755769 1210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:35.755836 1208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:35.756027 1206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:35.756948 1201 server_base.cc:1034] running on GCE node
I20260809 22:59:35.757597 1201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:35.758466 1201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:35.759680 1201 hybrid_clock.cc:648] HybridClock initialized: now 1786316375759578 us; error 70 us; skew 500 ppm
I20260809 22:59:35.759934 1201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:35 dist-test-slave-vxj2 krb5kdc[750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316375, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:35.763144 1201 init.cc:377] Logged in from keytab as oryx/127.24.31.195@KRBTEST.COM (short username oryx)
I20260809 22:59:35.764617 1201 webserver.cc:460] Webserver started at http://127.24.31.195:37743/ using document root <none> and password file <none>
I20260809 22:59:35.765368 1201 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:35.765458 1201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:35.765793 1201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:35.768725 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "412c681c4ac04123989843e747b439ff"
format_stamp: "Formatted at 2026-08-09 22:59:35 on dist-test-slave-vxj2"
server_key: "733b6c0a37fa8decda9167b266d17e83"
server_key_iv: "9eda4ea55f6ff5c9ba2bf1416e33c27e"
server_key_version: "encryptionkey@0"
I20260809 22:59:35.769536 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "412c681c4ac04123989843e747b439ff"
format_stamp: "Formatted at 2026-08-09 22:59:35 on dist-test-slave-vxj2"
server_key: "733b6c0a37fa8decda9167b266d17e83"
server_key_iv: "9eda4ea55f6ff5c9ba2bf1416e33c27e"
server_key_version: "encryptionkey@0"
I20260809 22:59:35.775098 1201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 22:59:35.778731 1217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:35.780076 1201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 22:59:35.780246 1201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "412c681c4ac04123989843e747b439ff"
format_stamp: "Formatted at 2026-08-09 22:59:35 on dist-test-slave-vxj2"
server_key: "733b6c0a37fa8decda9167b266d17e83"
server_key_iv: "9eda4ea55f6ff5c9ba2bf1416e33c27e"
server_key_version: "encryptionkey@0"
I20260809 22:59:35.780467 1201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:35.800828 1201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:35.804320 1038 heartbeater.cc:503] Master 127.24.31.254:44915 was elected leader, sending a full tablet report...
I20260809 22:59:35.805271 1201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:35.805540 1201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:35.808209 1201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:35.808321 1201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:35.808473 1201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:35.808550 1201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 22:59:35 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316375, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:35.833033 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:35.809009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:42349 (local address 127.24.31.254:44915)
0809 22:59:35.809207 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:35.809214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:35.810824 (+ 1610us) server_negotiation.cc:407] Connection header received
0809 22:59:35.813357 (+ 2533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:35.813366 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:35.813464 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:35.813547 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:35.816249 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.817407 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.819839 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.820411 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.820551 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:35.825793 (+ 5242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:35.825800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:35.825804 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:35.825848 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:35.828582 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:35.829439 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:35.829455 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:35.829457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:35.829647 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:35.830371 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:35.830378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:35.830385 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:35.830845 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:35.831030 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:35.832111 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0809 22:59:35.832469 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 22:59:35.833562 1227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:35.809626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44915 (local address 127.24.31.195:42349)
0809 22:59:35.810635 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:35.810680 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:35.812974 (+ 2294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:35.814107 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:35.814122 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:35.815022 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:35.816033 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:35.816066 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.817613 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.817618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:35.819511 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:35.819527 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.820908 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.820921 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:35.821263 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:35.822053 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:35.822076 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:35.825568 (+ 3492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:35.828729 (+ 3161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:35.828738 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:35.828754 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:35.829241 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:35.829795 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:35.829803 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:35.829812 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:35.830146 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:35.831016 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:35.831027 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:35.831520 (+ 493us) client_negotiation.cc:769] Sending connection context
0809 22:59:35.831913 (+ 393us) client_negotiation.cc:240] Negotiation successful
0809 22:59:35.832322 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":434,"threads_started":1}
I20260809 22:59:35.857774 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:35.839875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44915 (local address 127.24.31.195:38437)
0809 22:59:35.841766 (+ 1891us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:35.841794 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:35.841965 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:35.842694 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:35.842703 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:35.843653 (+ 950us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:35.844271 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:35.844304 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.845737 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.845742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:35.847145 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:35.847158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.848009 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.848022 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:35.848279 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:35.849000 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:35.849033 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:35.850428 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:35.853725 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:35.853731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:35.853736 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:35.854463 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:35.854925 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:35.854930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:35.854934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:35.855125 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:35.856026 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:35.856034 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:35.856355 (+ 321us) client_negotiation.cc:769] Sending connection context
0809 22:59:35.856561 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 22:59:35.857021 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1741,"thread_start_us":1529,"threads_started":1}
I20260809 22:59:35.857818 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:35.840096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:38437 (local address 127.24.31.254:44915)
0809 22:59:35.840327 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:35.840333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:35.841879 (+ 1546us) server_negotiation.cc:407] Connection header received
0809 22:59:35.842202 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:35.842210 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:35.842339 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:35.842461 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:35.844487 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.845555 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.847315 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.847769 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.847918 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:35.850656 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:35.850661 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:35.850665 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:35.850696 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:35.853539 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:35.854616 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:35.854623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:35.854628 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:35.854807 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:35.855279 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:35.855292 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:35.855297 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:35.855720 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:35.855943 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:35.857359 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0809 22:59:35.857551 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 22:59:35.864125 1201 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:41963
I20260809 22:59:35.864156 1349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:41963 every 8 connection(s)
I20260809 22:59:35.865386 1201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:35.867362 1350 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44915
I20260809 22:59:35.867621 1350 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:35.868259 1350 heartbeater.cc:511] Master 127.24.31.254:44915 requested a full tablet report, sending...
I20260809 22:59:35.869674 806 ts_manager.cc:194] Registered new tserver with Master: 412c681c4ac04123989843e747b439ff (127.24.31.195:41963)
I20260809 22:59:35.870540 806 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:42349
I20260809 22:59:35.870942 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 1201
I20260809 22:59:35.871167 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:59:35.871627 24703 external_mini_cluster.cc:1428] Setting key 591146201dd0a7c6f0bb4d984cfb54a9
I20260809 22:59:35.875000 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 22:59:35 dist-test-slave-vxj2 krb5kdc[750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316375, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 22:59:35 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:35 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:36.337729 1196 heartbeater.cc:503] Master 127.24.31.254:44915 was elected leader, sending a full tablet report...
I20260809 22:59:36.873306 1350 heartbeater.cc:503] Master 127.24.31.254:44915 was elected leader, sending a full tablet report...
W20260809 22:59:37.051092 1358 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.128s user 0.006s sys 0.010s
W20260809 22:59:37.052428 910 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:35.917483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.24.31.254:44915)
0809 22:59:35.917725 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:35.917733 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:35.917988 (+ 255us) server_negotiation.cc:407] Connection header received
0809 22:59:35.918247 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:35.918251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:35.918331 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:35.918441 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:35.919776 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.920522 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.921847 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:35.922272 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:35.922413 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.052075 (+1129662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33094: BlockingRecv error: recv got EOF from 127.0.0.1:33094 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 22:59:37 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316375, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:37.076162 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.058793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.24.31.254:44915)
0809 22:59:37.059014 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.059020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.059355 (+ 335us) server_negotiation.cc:407] Connection header received
0809 22:59:37.059505 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.059509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.059615 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.059721 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:37.060938 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.061769 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.062991 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.063489 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.063607 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.070484 (+ 6877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:37.070490 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:37.070493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:37.070527 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:37.073565 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:37.074231 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:37.074238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:37.074240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:37.074383 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:37.074827 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:37.074831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:37.074833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:37.075141 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:37.075385 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.075678 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.075876 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:59:37.080363 806 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:59:37.087654 806 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:59:37.110383 1373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.099562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.24.31.195:41963)
0809 22:59:37.100139 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.100152 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.101185 (+ 1033us) server_negotiation.cc:407] Connection header received
0809 22:59:37.101568 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.101578 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.101883 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.102078 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:37.103865 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.105344 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.107773 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.109265 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.109427 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.109631 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.109852 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.110036 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":267,"threads_started":1}
I20260809 22:59:37.110689 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.100007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53010 (local address 127.24.31.194:34911)
0809 22:59:37.100650 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.100659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.101325 (+ 666us) server_negotiation.cc:407] Connection header received
0809 22:59:37.101598 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.101607 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.101855 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.102078 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:37.103375 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.104883 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.107678 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.108889 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.109058 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.109300 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.110219 (+ 919us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.110389 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":338,"threads_started":1}
I20260809 22:59:37.111043 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.099423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:34911 (local address 127.0.0.1:53010)
0809 22:59:37.101209 (+ 1786us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.101227 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.101359 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.102376 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.102387 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.102464 (+ 77us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:37.103103 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.103121 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.105014 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.105019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.107479 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.107492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.109113 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.109119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.109350 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.109415 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.109556 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.109749 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1618,"spinlock_wait_cycles":2816,"thread_start_us":1471,"threads_started":1}
I20260809 22:59:37.111231 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.099295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:41963 (local address 127.0.0.1:47802)
0809 22:59:37.101006 (+ 1711us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.101062 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.101310 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.102392 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.102399 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.102478 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:37.103630 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.103643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.105453 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.105461 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.107600 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.107618 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.109478 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.109485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.109647 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.109677 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.110671 (+ 994us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.110816 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1339,"mutex_wait_us":1,"spinlock_wait_cycles":32384,"thread_start_us":1002,"threads_started":1}
I20260809 22:59:37.112429 1375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.101542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:42483 (local address 127.0.0.1:42132)
0809 22:59:37.103014 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.103030 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.103133 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.104691 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.104696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.104725 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:37.105202 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.105219 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.107113 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.107118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.109873 (+ 2755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.109885 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.111760 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.111766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.111920 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.111950 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.112045 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.112166 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1341,"thread_start_us":1188,"threads_started":1}
I20260809 22:59:37.112607 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.101626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42132 (local address 127.24.31.193:42483)
0809 22:59:37.104150 (+ 2524us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.104158 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.104186 (+ 28us) server_negotiation.cc:407] Connection header received
0809 22:59:37.104288 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.104296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.104499 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.104674 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:37.105395 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.106969 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.110061 (+ 3092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.111561 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.111672 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.111803 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.112131 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.112309 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2365,"thread_start_us":2209,"threads_started":1}
I20260809 22:59:37.115303 1118 tablet_service.cc:1467] Processing CreateTablet for tablet 5b20aa087dd64ed0bd20433a4a416635 (DEFAULT_TABLE table=test-table [id=9fc9080ae6b54d99b7b2c810052c4c1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:37.115432 1275 tablet_service.cc:1467] Processing CreateTablet for tablet 5b20aa087dd64ed0bd20433a4a416635 (DEFAULT_TABLE table=test-table [id=9fc9080ae6b54d99b7b2c810052c4c1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:37.115932 957 tablet_service.cc:1467] Processing CreateTablet for tablet 5b20aa087dd64ed0bd20433a4a416635 (DEFAULT_TABLE table=test-table [id=9fc9080ae6b54d99b7b2c810052c4c1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:37.116626 1118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b20aa087dd64ed0bd20433a4a416635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:37.116780 1275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b20aa087dd64ed0bd20433a4a416635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:37.117230 957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b20aa087dd64ed0bd20433a4a416635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:37.134418 1380 tablet_bootstrap.cc:492] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: Bootstrap starting.
I20260809 22:59:37.136133 1378 tablet_bootstrap.cc:492] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: Bootstrap starting.
I20260809 22:59:37.137828 1377 tablet_bootstrap.cc:492] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: Bootstrap starting.
I20260809 22:59:37.139770 1380 tablet_bootstrap.cc:654] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:37.139776 1378 tablet_bootstrap.cc:654] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:37.140956 1380 log.cc:824] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:37.141009 1378 log.cc:824] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:37.141068 1377 tablet_bootstrap.cc:654] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:37.142485 1377 log.cc:824] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:37.145802 1380 tablet_bootstrap.cc:492] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: No bootstrap required, opened a new log
I20260809 22:59:37.145968 1378 tablet_bootstrap.cc:492] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: No bootstrap required, opened a new log
I20260809 22:59:37.146051 1380 ts_tablet_manager.cc:1397] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.005s
I20260809 22:59:37.146273 1378 ts_tablet_manager.cc:1397] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260809 22:59:37.154284 1377 tablet_bootstrap.cc:492] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: No bootstrap required, opened a new log
I20260809 22:59:37.154558 1377 ts_tablet_manager.cc:1397] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.010s
I20260809 22:59:37.155510 1380 raft_consensus.cc:348] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } }
I20260809 22:59:37.155961 1380 raft_consensus.cc:374] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:37.156059 1380 raft_consensus.cc:729] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d3f03e4888498db89cfd644bcf693d, State: Initialized, Role: FOLLOWER
I20260809 22:59:37.156667 1380 consensus_queue.cc:260] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } }
I20260809 22:59:37.157934 1378 raft_consensus.cc:348] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } }
I20260809 22:59:37.158357 1378 raft_consensus.cc:374] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:37.158419 1380 ts_tablet_manager.cc:1428] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: Time spent starting tablet: real 0.012s user 0.005s sys 0.007s
I20260809 22:59:37.158447 1378 raft_consensus.cc:729] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412c681c4ac04123989843e747b439ff, State: Initialized, Role: FOLLOWER
I20260809 22:59:37.159039 1378 consensus_queue.cc:260] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } }
I20260809 22:59:37.160684 1378 ts_tablet_manager.cc:1428] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 22:59:37.164855 1377 raft_consensus.cc:348] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } }
I20260809 22:59:37.165376 1377 raft_consensus.cc:374] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:37.165517 1377 raft_consensus.cc:729] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87140d10272047d5a56cc9b5e6f28aad, State: Initialized, Role: FOLLOWER
I20260809 22:59:37.166195 1377 consensus_queue.cc:260] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } }
I20260809 22:59:37.167976 1377 ts_tablet_manager.cc:1428] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 22:59:37.268940 1389 raft_consensus.cc:482] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:37.269297 1389 raft_consensus.cc:504] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } }
I20260809 22:59:37.271162 1389 leader_election.cc:288] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 412c681c4ac04123989843e747b439ff (127.24.31.195:41963), 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483)
I20260809 22:59:37.278741 1373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.271606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:59737 (local address 127.24.31.195:41963)
0809 22:59:37.271855 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.271864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.272811 (+ 947us) server_negotiation.cc:407] Connection header received
0809 22:59:37.272979 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.273004 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.273128 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.273270 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:37.274036 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.274806 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.276842 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.277712 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.277856 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.277953 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.278336 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.278468 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 22:59:37.278755 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.271735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:43541 (local address 127.24.31.193:42483)
0809 22:59:37.271989 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.271998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.272644 (+ 646us) server_negotiation.cc:407] Connection header received
0809 22:59:37.272821 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.272826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.272925 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.273011 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:37.273852 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.274874 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.276901 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.277741 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.277937 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.278005 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.278383 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.278505 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 22:59:37.278913 1390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.271477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:42483 (local address 127.24.31.194:43541)
0809 22:59:37.272516 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.272546 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.272671 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.273084 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.273089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.273112 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:37.273654 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.273668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.275034 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.275038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.276702 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.276714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.278029 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.278034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.278203 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.278225 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.278334 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.278479 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":61,"thread_start_us":606,"threads_started":1}
I20260809 22:59:37.279368 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.271521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:41963 (local address 127.24.31.194:59737)
0809 22:59:37.272663 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.272682 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.272840 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.273311 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.273314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.273409 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:37.273844 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.273856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.274939 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.274948 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.276659 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.276692 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.277950 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.277956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.278166 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.278201 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.278285 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.278429 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"mutex_wait_us":12,"spinlock_wait_cycles":20352,"thread_start_us":716,"threads_started":1}
I20260809 22:59:37.280032 979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b20aa087dd64ed0bd20433a4a416635" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d3f03e4888498db89cfd644bcf693d" is_pre_election: true
I20260809 22:59:37.280381 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b20aa087dd64ed0bd20433a4a416635" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412c681c4ac04123989843e747b439ff" is_pre_election: true
I20260809 22:59:37.280526 979 raft_consensus.cc:2454] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 0.
I20260809 22:59:37.280846 1300 raft_consensus.cc:2454] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 0.
I20260809 22:59:37.281225 1065 leader_election.cc:302] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85d3f03e4888498db89cfd644bcf693d, 87140d10272047d5a56cc9b5e6f28aad; no voters:
I20260809 22:59:37.281714 1389 raft_consensus.cc:2781] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:37.281873 1389 raft_consensus.cc:482] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:37.281981 1389 raft_consensus.cc:3037] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:37.284241 1389 raft_consensus.cc:504] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } }
I20260809 22:59:37.284891 1389 leader_election.cc:288] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 election: Requested vote from peers 412c681c4ac04123989843e747b439ff (127.24.31.195:41963), 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483)
I20260809 22:59:37.285387 1300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b20aa087dd64ed0bd20433a4a416635" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412c681c4ac04123989843e747b439ff"
I20260809 22:59:37.285387 979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b20aa087dd64ed0bd20433a4a416635" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d3f03e4888498db89cfd644bcf693d"
I20260809 22:59:37.285595 979 raft_consensus.cc:3037] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:37.285595 1300 raft_consensus.cc:3037] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:37.288803 979 raft_consensus.cc:2454] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 1.
I20260809 22:59:37.288803 1300 raft_consensus.cc:2454] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 1.
I20260809 22:59:37.289238 1065 leader_election.cc:302] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85d3f03e4888498db89cfd644bcf693d, 87140d10272047d5a56cc9b5e6f28aad; no voters:
I20260809 22:59:37.289556 1389 raft_consensus.cc:2781] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:37.290031 1389 raft_consensus.cc:686] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 1 LEADER]: Becoming Leader. State: Replica: 87140d10272047d5a56cc9b5e6f28aad, State: Running, Role: LEADER
I20260809 22:59:37.290467 1389 consensus_queue.cc:237] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } }
I20260809 22:59:37.294997 806 catalog_manager.cc:5495] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad reported cstate change: term changed from 0 to 1, leader changed from <none> to 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194). New cstate: current_term: 1 leader_uuid: "87140d10272047d5a56cc9b5e6f28aad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } health_report { overall_health: UNKNOWN } } }
W20260809 22:59:37.298873 1039 tablet.cc:2369] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 22:59:37.330773 1197 tablet.cc:2369] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:59:37.354801 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.349255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53016 (local address 127.24.31.194:34911)
0809 22:59:37.349492 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.349498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.349518 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:59:37.349690 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.349694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.349782 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.349932 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:37.350306 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.351191 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.352598 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.352998 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.353149 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.353641 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:37.354187 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:37.354363 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.354453 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.354535 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 22:59:37.363049 979 raft_consensus.cc:1261] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Refusing update from remote peer 87140d10272047d5a56cc9b5e6f28aad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:37.363212 1300 raft_consensus.cc:1261] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Refusing update from remote peer 87140d10272047d5a56cc9b5e6f28aad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:37.364078 1392 consensus_queue.cc:1035] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:37.364403 1389 consensus_queue.cc:1035] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [LEADER]: Connected to new peer: Peer: permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 22:59:37.367983 1351 tablet.cc:2369] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 22:59:37.377430 1397 mvcc.cc:205] Tried to move back new op lower bound from 7316751881669156864 to 7316751881384935424. Current Snapshot: MvccSnapshot[applied={T|T < 7316751881669156864}]
I20260809 22:59:37.380789 1396 mvcc.cc:205] Tried to move back new op lower bound from 7316751881669156864 to 7316751881384935424. Current Snapshot: MvccSnapshot[applied={T|T < 7316751881669156864}]
I20260809 22:59:37.398345 806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:37.398792 806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:37.405629 806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e: Sending DeleteTablet for 3 replicas of tablet 5b20aa087dd64ed0bd20433a4a416635
I20260809 22:59:37.406854 1275 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b20aa087dd64ed0bd20433a4a416635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:37 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:47802
I20260809 22:59:37.406929 1118 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b20aa087dd64ed0bd20433a4a416635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:37 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:53010
I20260809 22:59:37.407603 957 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b20aa087dd64ed0bd20433a4a416635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:37 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:42132
I20260809 22:59:37.407676 956 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b20aa087dd64ed0bd20433a4a416635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:37 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:42132
I20260809 22:59:37.408533 1406 tablet_replica.cc:330] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: stopping tablet replica
I20260809 22:59:37.409097 1406 raft_consensus.cc:2229] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:37.409711 1406 raft_consensus.cc:2258] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:37.409688 806 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:59:37.410430 806 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:59:37.412639 1406 ts_tablet_manager.cc:1905] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:37.417976 1405 tablet_replica.cc:330] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: stopping tablet replica
I20260809 22:59:37.418444 1405 raft_consensus.cc:2229] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:59:37.419173 1405 raft_consensus.cc:2258] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:37.420015 1404 tablet_replica.cc:330] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: stopping tablet replica
I20260809 22:59:37.420058 1116 tablet_service.cc:1467] Processing CreateTablet for tablet a6944a5399884165a6a7074472e38b69 (DEFAULT_TABLE table=test-table [id=07f87ff1682c46fd850cdf5a4382100a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:37.420070 957 tablet_service.cc:1467] Processing CreateTablet for tablet a6944a5399884165a6a7074472e38b69 (DEFAULT_TABLE table=test-table [id=07f87ff1682c46fd850cdf5a4382100a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:37.420527 957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6944a5399884165a6a7074472e38b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:37.420583 1404 raft_consensus.cc:2229] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:37.420778 1116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6944a5399884165a6a7074472e38b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:37.421092 1404 raft_consensus.cc:2258] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:37.423171 1404 ts_tablet_manager.cc:1905] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:37.425800 1272 tablet_service.cc:1467] Processing CreateTablet for tablet a6944a5399884165a6a7074472e38b69 (DEFAULT_TABLE table=test-table [id=07f87ff1682c46fd850cdf5a4382100a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:37.426234 1272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6944a5399884165a6a7074472e38b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:37.430074 1404 ts_tablet_manager.cc:1918] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:37.430243 1404 log.cc:1196] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/5b20aa087dd64ed0bd20433a4a416635
I20260809 22:59:37.430728 1380 tablet_bootstrap.cc:492] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: Bootstrap starting.
I20260809 22:59:37.432534 1380 tablet_bootstrap.cc:654] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:37.432761 1405 ts_tablet_manager.cc:1905] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:37.433575 1404 ts_tablet_manager.cc:1939] T 5b20aa087dd64ed0bd20433a4a416635 P 412c681c4ac04123989843e747b439ff: Deleting consensus metadata
I20260809 22:59:37.436713 788 catalog_manager.cc:4841] TS 412c681c4ac04123989843e747b439ff (127.24.31.195:41963): tablet 5b20aa087dd64ed0bd20433a4a416635 (table test-table [id=9fc9080ae6b54d99b7b2c810052c4c1d]) successfully deleted
I20260809 22:59:37.439810 1377 tablet_bootstrap.cc:492] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: Bootstrap starting.
I20260809 22:59:37.441692 1377 tablet_bootstrap.cc:654] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:37.442536 1378 tablet_bootstrap.cc:492] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: Bootstrap starting.
I20260809 22:59:37.444571 1378 tablet_bootstrap.cc:654] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:37.445695 1377 tablet_bootstrap.cc:492] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: No bootstrap required, opened a new log
I20260809 22:59:37.445919 1377 ts_tablet_manager.cc:1397] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 22:59:37.446852 1377 raft_consensus.cc:348] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.447196 1377 raft_consensus.cc:374] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:37.447381 1377 raft_consensus.cc:729] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87140d10272047d5a56cc9b5e6f28aad, State: Initialized, Role: FOLLOWER
I20260809 22:59:37.447731 1378 tablet_bootstrap.cc:492] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: No bootstrap required, opened a new log
I20260809 22:59:37.447876 1378 ts_tablet_manager.cc:1397] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 22:59:37.447832 1377 consensus_queue.cc:260] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.448557 1377 ts_tablet_manager.cc:1428] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 22:59:37.449469 1378 raft_consensus.cc:348] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.449692 1378 raft_consensus.cc:374] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:37.449800 1378 raft_consensus.cc:729] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412c681c4ac04123989843e747b439ff, State: Initialized, Role: FOLLOWER
I20260809 22:59:37.450223 1378 consensus_queue.cc:260] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.450932 1378 ts_tablet_manager.cc:1428] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 22:59:37.456887 1406 ts_tablet_manager.cc:1918] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:37.457140 1406 log.cc:1196] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/5b20aa087dd64ed0bd20433a4a416635
I20260809 22:59:37.459110 1406 ts_tablet_manager.cc:1939] T 5b20aa087dd64ed0bd20433a4a416635 P 85d3f03e4888498db89cfd644bcf693d: Deleting consensus metadata
I20260809 22:59:37.461494 790 catalog_manager.cc:4841] TS 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483): tablet 5b20aa087dd64ed0bd20433a4a416635 (table test-table [id=9fc9080ae6b54d99b7b2c810052c4c1d]) successfully deleted
W20260809 22:59:37.462167 790 catalog_manager.cc:4808] TS 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483): delete failed for tablet 5b20aa087dd64ed0bd20433a4a416635 because the tablet was not found. No further retry: Not found: Tablet not found: 5b20aa087dd64ed0bd20433a4a416635
I20260809 22:59:37.464104 1405 ts_tablet_manager.cc:1918] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:37.464291 1405 log.cc:1196] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/5b20aa087dd64ed0bd20433a4a416635
I20260809 22:59:37.464857 1405 ts_tablet_manager.cc:1939] T 5b20aa087dd64ed0bd20433a4a416635 P 87140d10272047d5a56cc9b5e6f28aad: Deleting consensus metadata
I20260809 22:59:37.466696 790 catalog_manager.cc:4841] TS 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194:34911): tablet 5b20aa087dd64ed0bd20433a4a416635 (table test-table [id=9fc9080ae6b54d99b7b2c810052c4c1d]) successfully deleted
I20260809 22:59:37.467767 1380 tablet_bootstrap.cc:492] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: No bootstrap required, opened a new log
I20260809 22:59:37.467924 1380 ts_tablet_manager.cc:1397] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: Time spent bootstrapping tablet: real 0.037s user 0.019s sys 0.012s
I20260809 22:59:37.468590 1380 raft_consensus.cc:348] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.468761 1380 raft_consensus.cc:374] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:37.468884 1380 raft_consensus.cc:729] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d3f03e4888498db89cfd644bcf693d, State: Initialized, Role: FOLLOWER
I20260809 22:59:37.469206 1380 consensus_queue.cc:260] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.469789 1380 ts_tablet_manager.cc:1428] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:59:37.509624 1386 raft_consensus.cc:482] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:37.509922 1386 raft_consensus.cc:504] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.511762 1386 leader_election.cc:288] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483), 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194:34911)
I20260809 22:59:37.519977 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.512193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:60185 (local address 127.24.31.193:42483)
0809 22:59:37.512418 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.512425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.512850 (+ 425us) server_negotiation.cc:407] Connection header received
0809 22:59:37.513514 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.513523 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.513640 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.513717 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:37.514782 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.515855 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.518210 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.519005 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.519210 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.519297 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.519575 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.519697 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 22:59:37.520010 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.511969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:42483 (local address 127.24.31.195:60185)
0809 22:59:37.512752 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.512773 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.512969 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.513996 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.514004 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.514043 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:37.514527 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.514543 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.516007 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.516014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.518059 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.518071 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.519238 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.519244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.519404 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.519428 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.519537 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.519674 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":473,"threads_started":1}
I20260809 22:59:37.520758 977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6944a5399884165a6a7074472e38b69" candidate_uuid: "412c681c4ac04123989843e747b439ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d3f03e4888498db89cfd644bcf693d" is_pre_election: true
I20260809 22:59:37.521088 977 raft_consensus.cc:2454] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 412c681c4ac04123989843e747b439ff in term 0.
I20260809 22:59:37.521692 1220 leader_election.cc:302] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412c681c4ac04123989843e747b439ff, 85d3f03e4888498db89cfd644bcf693d; no voters:
I20260809 22:59:37.522192 1386 raft_consensus.cc:2781] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:37.522330 1386 raft_consensus.cc:482] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:37.522418 1386 raft_consensus.cc:3037] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:37.522467 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.512919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:34911 (local address 127.24.31.195:33933)
0809 22:59:37.515106 (+ 2187us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.515125 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.515282 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.515653 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.515658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.515685 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:37.516067 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.516079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.517612 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.517618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.519699 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.519715 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.521530 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.521536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.521732 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.521763 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.521922 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.522066 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2045,"thread_start_us":1850,"threads_started":1}
I20260809 22:59:37.522475 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.513212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:33933 (local address 127.24.31.194:34911)
0809 22:59:37.513403 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.513409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.515235 (+ 1826us) server_negotiation.cc:407] Connection header received
0809 22:59:37.515426 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.515431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.515509 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.515608 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:37.516282 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.517468 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.519884 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.521312 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.521476 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.521548 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.521973 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.522139 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 22:59:37.523880 1139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6944a5399884165a6a7074472e38b69" candidate_uuid: "412c681c4ac04123989843e747b439ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87140d10272047d5a56cc9b5e6f28aad" is_pre_election: true
I20260809 22:59:37.524353 1139 raft_consensus.cc:2454] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 412c681c4ac04123989843e747b439ff in term 0.
I20260809 22:59:37.524977 1386 raft_consensus.cc:504] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.525583 1386 leader_election.cc:288] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [CANDIDATE]: Term 1 election: Requested vote from peers 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483), 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194:34911)
I20260809 22:59:37.526120 1139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6944a5399884165a6a7074472e38b69" candidate_uuid: "412c681c4ac04123989843e747b439ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87140d10272047d5a56cc9b5e6f28aad"
I20260809 22:59:37.526180 977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6944a5399884165a6a7074472e38b69" candidate_uuid: "412c681c4ac04123989843e747b439ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d3f03e4888498db89cfd644bcf693d"
I20260809 22:59:37.526324 1139 raft_consensus.cc:3037] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:37.526415 977 raft_consensus.cc:3037] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:37.528604 1139 raft_consensus.cc:2454] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 412c681c4ac04123989843e747b439ff in term 1.
I20260809 22:59:37.528987 1220 leader_election.cc:302] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412c681c4ac04123989843e747b439ff, 87140d10272047d5a56cc9b5e6f28aad; no voters:
I20260809 22:59:37.529297 1386 raft_consensus.cc:2781] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:37.529516 977 raft_consensus.cc:2454] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 412c681c4ac04123989843e747b439ff in term 1.
I20260809 22:59:37.529937 1386 raft_consensus.cc:686] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 1 LEADER]: Becoming Leader. State: Replica: 412c681c4ac04123989843e747b439ff, State: Running, Role: LEADER
I20260809 22:59:37.530344 1386 consensus_queue.cc:237] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.534307 805 catalog_manager.cc:5495] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 412c681c4ac04123989843e747b439ff (127.24.31.195). New cstate: current_term: 1 leader_uuid: "412c681c4ac04123989843e747b439ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } health_report { overall_health: HEALTHY } } }
Aug 09 22:59:37 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316374, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.254@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:37.593791 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.579330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44915 (local address 127.0.0.1:33110)
0809 22:59:37.580104 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.580131 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.580307 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.580729 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.580733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.581575 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:37.581820 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.581829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.582961 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.582969 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.584014 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.584026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.584772 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.584778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.584966 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.585700 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:37.585719 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:37.588208 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:37.591181 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:37.591197 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:37.591204 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:37.591614 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:37.591982 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:37.591988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:37.591992 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:37.592125 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:37.592627 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:37.592635 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:37.592812 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.592972 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.593299 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":449,"threads_started":1}
I20260809 22:59:37.593811 1424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.579404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33110 (local address 127.24.31.254:44915)
0809 22:59:37.580484 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.580488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.580504 (+ 16us) server_negotiation.cc:407] Connection header received
0809 22:59:37.580540 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.580543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.580594 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.580705 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:37.582005 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.582837 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.584177 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.584587 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.584728 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.588412 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:37.588417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:37.588421 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:37.588448 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:37.591053 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:37.591724 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:37.591731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:37.591735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:37.591860 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:37.592224 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:37.592231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:37.592237 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:37.592500 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:37.592653 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.593096 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.593446 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":963,"thread_start_us":772,"threads_started":1}
I20260809 22:59:37.599506 805 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:33110:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 22:59:37.600788 805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:59:37.608140 957 tablet_service.cc:1467] Processing CreateTablet for tablet 633797ab31f246cab6f1c66494339dce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09549905256248c7b013d590e03a601b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:59:37.608171 1116 tablet_service.cc:1467] Processing CreateTablet for tablet 633797ab31f246cab6f1c66494339dce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09549905256248c7b013d590e03a601b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:59:37.608362 1272 tablet_service.cc:1467] Processing CreateTablet for tablet 633797ab31f246cab6f1c66494339dce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09549905256248c7b013d590e03a601b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 22:59:37.608764 957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 633797ab31f246cab6f1c66494339dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:37.608773 1116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 633797ab31f246cab6f1c66494339dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:37.608841 1272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 633797ab31f246cab6f1c66494339dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:37.615967 1380 tablet_bootstrap.cc:492] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: Bootstrap starting.
I20260809 22:59:37.616325 1378 tablet_bootstrap.cc:492] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: Bootstrap starting.
I20260809 22:59:37.617802 1380 tablet_bootstrap.cc:654] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:37.617890 1378 tablet_bootstrap.cc:654] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:37.618858 1377 tablet_bootstrap.cc:492] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: Bootstrap starting.
I20260809 22:59:37.619820 1378 tablet_bootstrap.cc:492] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: No bootstrap required, opened a new log
I20260809 22:59:37.619992 1378 ts_tablet_manager.cc:1397] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 22:59:37.620537 1378 raft_consensus.cc:348] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.620684 1377 tablet_bootstrap.cc:654] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:37.620781 1378 raft_consensus.cc:374] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:37.620882 1378 raft_consensus.cc:729] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412c681c4ac04123989843e747b439ff, State: Initialized, Role: FOLLOWER
I20260809 22:59:37.621140 1378 consensus_queue.cc:260] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.621582 1386 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.621893 1386 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:59:37.622346 1378 ts_tablet_manager.cc:1428] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 22:59:37.627054 1377 tablet_bootstrap.cc:492] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: No bootstrap required, opened a new log
I20260809 22:59:37.627246 1377 ts_tablet_manager.cc:1397] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.002s
I20260809 22:59:37.628021 1377 raft_consensus.cc:348] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.628293 1377 raft_consensus.cc:374] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:37.628391 1377 raft_consensus.cc:729] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87140d10272047d5a56cc9b5e6f28aad, State: Initialized, Role: FOLLOWER
I20260809 22:59:37.628651 1377 consensus_queue.cc:260] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.629212 1389 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.629370 1377 ts_tablet_manager.cc:1428] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 22:59:37.629482 1389 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:59:37.631323 1380 tablet_bootstrap.cc:492] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: No bootstrap required, opened a new log
I20260809 22:59:37.631567 1380 ts_tablet_manager.cc:1397] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.001s
I20260809 22:59:37.632371 1380 raft_consensus.cc:348] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.632673 1380 raft_consensus.cc:374] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:37.632776 1380 raft_consensus.cc:729] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d3f03e4888498db89cfd644bcf693d, State: Initialized, Role: FOLLOWER
I20260809 22:59:37.633045 1380 consensus_queue.cc:260] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.633795 1380 ts_tablet_manager.cc:1428] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 22:59:37.633602 1385 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.634017 1385 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:59:37.822214 1385 raft_consensus.cc:482] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:37.822463 1385 raft_consensus.cc:504] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.824218 1385 leader_election.cc:288] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194:34911), 412c681c4ac04123989843e747b439ff (127.24.31.195:41963)
I20260809 22:59:37.832690 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.824616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:51117 (local address 127.24.31.194:34911)
0809 22:59:37.825820 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.825827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.825850 (+ 23us) server_negotiation.cc:407] Connection header received
0809 22:59:37.825931 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.825937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.826018 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.826114 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:37.826996 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.827853 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.830422 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.831696 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.831853 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.832210 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.832276 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.832423 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1022}
I20260809 22:59:37.832705 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.824664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:41963 (local address 127.24.31.193:49233)
0809 22:59:37.825588 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.825606 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.825709 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.826441 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.826445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.826491 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:37.827021 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.827038 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.828545 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.828555 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.830602 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.830612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.831782 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.831788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.831981 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.832013 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.832102 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.832229 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":53,"thread_start_us":789,"threads_started":1}
I20260809 22:59:37.833067 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.825186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:49233 (local address 127.24.31.195:41963)
0809 22:59:37.826141 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.826147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.826166 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:59:37.826233 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.826239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.826308 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.826419 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 22:59:37.827212 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.828398 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.830746 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.831552 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.831741 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.831839 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.832705 (+ 866us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.832869 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":709,"threads_started":1}
I20260809 22:59:37.833283 1429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.824432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:34911 (local address 127.24.31.193:51117)
0809 22:59:37.825294 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.825318 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.825477 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.826166 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.826171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.826203 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 22:59:37.826682 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.826700 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.827989 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.827997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.830218 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.830238 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.831996 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.832001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.832155 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.832182 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.832239 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.832344 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":1,"thread_start_us":481,"threads_started":1}
I20260809 22:59:37.833638 1299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633797ab31f246cab6f1c66494339dce" candidate_uuid: "85d3f03e4888498db89cfd644bcf693d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412c681c4ac04123989843e747b439ff" is_pre_election: true
I20260809 22:59:37.833953 1299 raft_consensus.cc:2454] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85d3f03e4888498db89cfd644bcf693d in term 0.
I20260809 22:59:37.834120 1139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633797ab31f246cab6f1c66494339dce" candidate_uuid: "85d3f03e4888498db89cfd644bcf693d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87140d10272047d5a56cc9b5e6f28aad" is_pre_election: true
I20260809 22:59:37.834403 1139 raft_consensus.cc:2454] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85d3f03e4888498db89cfd644bcf693d in term 0.
I20260809 22:59:37.834471 902 leader_election.cc:302] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412c681c4ac04123989843e747b439ff, 85d3f03e4888498db89cfd644bcf693d; no voters:
I20260809 22:59:37.834892 1385 raft_consensus.cc:2781] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:37.835028 1385 raft_consensus.cc:482] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:37.835120 1385 raft_consensus.cc:3037] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:37.837538 1385 raft_consensus.cc:504] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.838143 1385 leader_election.cc:288] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [CANDIDATE]: Term 1 election: Requested vote from peers 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194:34911), 412c681c4ac04123989843e747b439ff (127.24.31.195:41963)
I20260809 22:59:37.838603 1139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633797ab31f246cab6f1c66494339dce" candidate_uuid: "85d3f03e4888498db89cfd644bcf693d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87140d10272047d5a56cc9b5e6f28aad"
I20260809 22:59:37.838668 1299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "633797ab31f246cab6f1c66494339dce" candidate_uuid: "85d3f03e4888498db89cfd644bcf693d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412c681c4ac04123989843e747b439ff"
I20260809 22:59:37.838871 1299 raft_consensus.cc:3037] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:37.838899 1139 raft_consensus.cc:3037] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:37.841986 1139 raft_consensus.cc:2454] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85d3f03e4888498db89cfd644bcf693d in term 1.
I20260809 22:59:37.842010 1299 raft_consensus.cc:2454] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85d3f03e4888498db89cfd644bcf693d in term 1.
I20260809 22:59:37.842397 902 leader_election.cc:302] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412c681c4ac04123989843e747b439ff, 85d3f03e4888498db89cfd644bcf693d; no voters:
I20260809 22:59:37.842657 1385 raft_consensus.cc:2781] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:37.843139 1385 raft_consensus.cc:686] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [term 1 LEADER]: Becoming Leader. State: Replica: 85d3f03e4888498db89cfd644bcf693d, State: Running, Role: LEADER
I20260809 22:59:37.843551 1385 consensus_queue.cc:237] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:37.845458 1432 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: TxnStatusTablet state changed. Reason: New leader 85d3f03e4888498db89cfd644bcf693d. Latest consensus state: current_term: 1 leader_uuid: "85d3f03e4888498db89cfd644bcf693d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.845914 1432 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:59:37.847641 805 catalog_manager.cc:5495] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d reported cstate change: term changed from 0 to 1, leader changed from <none> to 85d3f03e4888498db89cfd644bcf693d (127.24.31.193). New cstate: current_term: 1 leader_uuid: "85d3f03e4888498db89cfd644bcf693d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } health_report { overall_health: UNKNOWN } } }
I20260809 22:59:37.848676 1434 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 22:59:37.849732 1434 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 22:59:37.876765 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.869258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42140 (local address 127.24.31.193:42483)
0809 22:59:37.869439 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.869445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.869526 (+ 81us) server_negotiation.cc:407] Connection header received
0809 22:59:37.869798 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.869804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.869934 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.870090 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:37.870509 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.871648 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.873692 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.874463 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.874586 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.875247 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:37.875986 (+ 739us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:37.876136 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.876294 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.876405 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 22:59:37.877280 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.869096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:42483 (local address 127.0.0.1:42140)
0809 22:59:37.869436 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.869454 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.869587 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.870141 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.870148 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.870164 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:59:37.870369 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.870378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.871796 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.871801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.873525 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.873540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.874656 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.874662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.874903 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.874979 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:59:37.876115 (+ 1136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:59:37.876140 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.876458 (+ 318us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.876666 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260809 22:59:37.882028 1299 raft_consensus.cc:1261] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Refusing update from remote peer 85d3f03e4888498db89cfd644bcf693d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:37.882761 1139 raft_consensus.cc:1261] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Refusing update from remote peer 85d3f03e4888498db89cfd644bcf693d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:37.882721 1432 consensus_queue.cc:1035] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [LEADER]: Connected to new peer: Peer: permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:37.883687 1432 consensus_queue.cc:1035] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d [LEADER]: Connected to new peer: Peer: permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:37.899431 1386 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: TxnStatusTablet state changed. Reason: New leader 85d3f03e4888498db89cfd644bcf693d. Latest consensus state: current_term: 1 leader_uuid: "85d3f03e4888498db89cfd644bcf693d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.899758 1386 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:59:37.901131 1385 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "85d3f03e4888498db89cfd644bcf693d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.901494 1385 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:59:37.903959 1389 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: TxnStatusTablet state changed. Reason: New leader 85d3f03e4888498db89cfd644bcf693d. Latest consensus state: current_term: 1 leader_uuid: "85d3f03e4888498db89cfd644bcf693d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.904234 1389 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:59:37.904261 1432 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "85d3f03e4888498db89cfd644bcf693d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.904510 1432 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 85d3f03e4888498db89cfd644bcf693d: This TxnStatusTablet replica's current role is: LEADER
I20260809 22:59:37.914264 1386 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "85d3f03e4888498db89cfd644bcf693d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.914659 1386 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 412c681c4ac04123989843e747b439ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:59:37.918838 1389 tablet_replica.cc:439] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "85d3f03e4888498db89cfd644bcf693d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } } }
I20260809 22:59:37.919113 1389 tablet_replica.cc:442] T 633797ab31f246cab6f1c66494339dce P 87140d10272047d5a56cc9b5e6f28aad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 22:59:37.925927 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.917378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.24.31.195:41963)
0809 22:59:37.917820 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.917830 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.918817 (+ 987us) server_negotiation.cc:407] Connection header received
0809 22:59:37.919555 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.919562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.919700 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.919866 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:37.920439 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.921414 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.923382 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.923775 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.923866 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.924493 (+ 627us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:37.925119 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:37.925320 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.925490 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.925561 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270}
I20260809 22:59:37.930738 1386 consensus_queue.cc:1035] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:37.939476 1415 consensus_queue.cc:1035] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:37.954540 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.947622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:33109 (local address 127.24.31.193:42483)
0809 22:59:37.947787 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.947793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.948465 (+ 672us) server_negotiation.cc:407] Connection header received
0809 22:59:37.948736 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.948743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.948842 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.948954 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:37.949529 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.950417 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.952440 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.953087 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.953316 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.953831 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:37.953918 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:37.954014 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:37.954173 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 22:59:37.954252 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 22:59:37.954931 1452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.947643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:42483 (local address 127.24.31.195:33109)
0809 22:59:37.948357 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.948379 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.948584 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.948984 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.948990 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.949003 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:59:37.949359 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.949384 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.950564 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.950570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.952293 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.952310 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.953327 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.953334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:37.953548 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:37.953619 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:59:37.954022 (+ 403us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:59:37.954034 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 22:59:37.954119 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 22:59:37.954279 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":379,"threads_started":1}
I20260809 22:59:38.003186 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.996734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:53483 (local address 127.24.31.195:41963)
0809 22:59:37.996904 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:37.996909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:37.996929 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:59:37.996994 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:37.996999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:37.997103 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:37.997282 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:37.998210 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.998962 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.001423 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.001865 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.001995 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.002451 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:38.002560 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:38.002657 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:38.002788 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 22:59:38.002867 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 22:59:38.003530 1457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:37.995443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:41963 (local address 127.24.31.193:53483)
0809 22:59:37.996189 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:37.996210 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:37.996354 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:37.997761 (+ 1407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:37.997765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:37.997774 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:59:37.998055 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:37.998062 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:37.999087 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:37.999095 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:38.001288 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:38.001301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.002055 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.002062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:38.002219 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.002279 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:59:38.002663 (+ 384us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:59:38.002676 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 22:59:38.002766 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 22:59:38.002904 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":1942,"threads_started":1}
I20260809 22:59:38.042861 805 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:38.043123 805 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:38.046638 805 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e: Sending DeleteTablet for 3 replicas of tablet a6944a5399884165a6a7074472e38b69
I20260809 22:59:38.047986 957 tablet_service.cc:1514] Processing DeleteTablet for tablet a6944a5399884165a6a7074472e38b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:38 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:42132
I20260809 22:59:38.048173 1116 tablet_service.cc:1514] Processing DeleteTablet for tablet a6944a5399884165a6a7074472e38b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:38 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:53010
I20260809 22:59:38.049108 1272 tablet_service.cc:1514] Processing DeleteTablet for tablet a6944a5399884165a6a7074472e38b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:38 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:47802
I20260809 22:59:38.070113 1462 tablet_replica.cc:330] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: stopping tablet replica
I20260809 22:59:38.071848 1462 raft_consensus.cc:2229] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:38.072229 1462 raft_consensus.cc:2258] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:38.073756 1462 ts_tablet_manager.cc:1905] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:38.074216 1463 tablet_replica.cc:330] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: stopping tablet replica
I20260809 22:59:38.074440 1463 raft_consensus.cc:2229] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:59:38.075279 1463 raft_consensus.cc:2258] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:38.076459 1463 ts_tablet_manager.cc:1905] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:38.081630 1461 tablet_replica.cc:330] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: stopping tablet replica
I20260809 22:59:38.081900 1461 raft_consensus.cc:2229] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:38.082680 1461 raft_consensus.cc:2258] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:38.084208 1463 ts_tablet_manager.cc:1918] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:59:38.084349 1463 log.cc:1196] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/a6944a5399884165a6a7074472e38b69
I20260809 22:59:38.084930 1463 ts_tablet_manager.cc:1939] T a6944a5399884165a6a7074472e38b69 P 412c681c4ac04123989843e747b439ff: Deleting consensus metadata
I20260809 22:59:38.085170 1461 ts_tablet_manager.cc:1905] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:38.086073 788 catalog_manager.cc:4841] TS 412c681c4ac04123989843e747b439ff (127.24.31.195:41963): tablet a6944a5399884165a6a7074472e38b69 (table test-table [id=07f87ff1682c46fd850cdf5a4382100a]) successfully deleted
I20260809 22:59:38.087711 1462 ts_tablet_manager.cc:1918] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:59:38.094689 1461 ts_tablet_manager.cc:1918] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:59:38.094834 1461 log.cc:1196] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/a6944a5399884165a6a7074472e38b69
I20260809 22:59:38.095340 1461 ts_tablet_manager.cc:1939] T a6944a5399884165a6a7074472e38b69 P 85d3f03e4888498db89cfd644bcf693d: Deleting consensus metadata
I20260809 22:59:38.096493 790 catalog_manager.cc:4841] TS 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483): tablet a6944a5399884165a6a7074472e38b69 (table test-table [id=07f87ff1682c46fd850cdf5a4382100a]) successfully deleted
I20260809 22:59:38.096750 1462 log.cc:1196] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/a6944a5399884165a6a7074472e38b69
I20260809 22:59:38.097360 1462 ts_tablet_manager.cc:1939] T a6944a5399884165a6a7074472e38b69 P 87140d10272047d5a56cc9b5e6f28aad: Deleting consensus metadata
I20260809 22:59:38.098495 790 catalog_manager.cc:4841] TS 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194:34911): tablet a6944a5399884165a6a7074472e38b69 (table test-table [id=07f87ff1682c46fd850cdf5a4382100a]) successfully deleted
Aug 09 22:59:38 dist-test-slave-vxj2 krb5kdc[750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 22:59:38 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:38 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:59:38.190852 1471 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:38.160063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.24.31.254:44915)
0809 22:59:38.175734 (+ 15671us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:38.175741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:38.175763 (+ 22us) server_negotiation.cc:407] Connection header received
0809 22:59:38.175829 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:38.175834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:38.175914 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:38.176051 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:38.177927 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.179062 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.181023 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.181798 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.181938 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.190520 (+ 8582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33114: BlockingRecv error: recv got EOF from 127.0.0.1:33114 (error 108)
Metrics: {"server-negotiator.queue_time_us":15428,"thread_start_us":15456,"threads_started":1}
Aug 09 22:59:38 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM
I20260809 22:59:38.211941 1471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:38.192897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.24.31.254:44915)
0809 22:59:38.193069 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:38.193077 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:38.193768 (+ 691us) server_negotiation.cc:407] Connection header received
0809 22:59:38.193954 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:38.193960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:38.194045 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:38.194154 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:38.195683 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.196845 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.198786 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.199420 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.199548 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.204661 (+ 5113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:38.204670 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:38.204723 (+ 53us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:38.204769 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:38.208599 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:38.209513 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:38.209521 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:38.209526 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:38.209734 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:38.210276 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:38.210281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:38.210285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:38.210647 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:38.210988 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:38.211348 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 22:59:38.211584 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 22:59:38.216305 805 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 22:59:38.217044 805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:59:38.225781 1116 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3d3b678a5044b8ac34d817f31e4352 (DEFAULT_TABLE table=test-table [id=5ecdd5faa75e4d51b84ea455ea047680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:38.225915 1272 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3d3b678a5044b8ac34d817f31e4352 (DEFAULT_TABLE table=test-table [id=5ecdd5faa75e4d51b84ea455ea047680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:38.226351 1116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3d3b678a5044b8ac34d817f31e4352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:38.226410 1272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3d3b678a5044b8ac34d817f31e4352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:38.233877 957 tablet_service.cc:1467] Processing CreateTablet for tablet 1f3d3b678a5044b8ac34d817f31e4352 (DEFAULT_TABLE table=test-table [id=5ecdd5faa75e4d51b84ea455ea047680]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:38.234436 957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f3d3b678a5044b8ac34d817f31e4352. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:38.239073 1478 tablet_bootstrap.cc:492] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: Bootstrap starting.
I20260809 22:59:38.241755 1478 tablet_bootstrap.cc:654] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:38.243307 1480 tablet_bootstrap.cc:492] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: Bootstrap starting.
I20260809 22:59:38.244702 1482 tablet_bootstrap.cc:492] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: Bootstrap starting.
I20260809 22:59:38.245564 1480 tablet_bootstrap.cc:654] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:38.247530 1478 tablet_bootstrap.cc:492] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: No bootstrap required, opened a new log
I20260809 22:59:38.247712 1478 ts_tablet_manager.cc:1397] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260809 22:59:38.248705 1478 raft_consensus.cc:348] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.249054 1478 raft_consensus.cc:374] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:38.249169 1478 raft_consensus.cc:729] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87140d10272047d5a56cc9b5e6f28aad, State: Initialized, Role: FOLLOWER
I20260809 22:59:38.249483 1478 consensus_queue.cc:260] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.250895 1478 ts_tablet_manager.cc:1428] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:38.252079 1482 tablet_bootstrap.cc:654] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:38.255600 1480 tablet_bootstrap.cc:492] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: No bootstrap required, opened a new log
I20260809 22:59:38.255856 1480 ts_tablet_manager.cc:1397] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260809 22:59:38.256856 1480 raft_consensus.cc:348] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.257179 1480 raft_consensus.cc:374] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:38.257303 1480 raft_consensus.cc:729] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412c681c4ac04123989843e747b439ff, State: Initialized, Role: FOLLOWER
I20260809 22:59:38.257575 1480 consensus_queue.cc:260] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.258203 1480 ts_tablet_manager.cc:1428] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 22:59:38.259301 1482 tablet_bootstrap.cc:492] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: No bootstrap required, opened a new log
I20260809 22:59:38.259457 1482 ts_tablet_manager.cc:1397] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260809 22:59:38.260365 1482 raft_consensus.cc:348] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.260663 1482 raft_consensus.cc:374] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:38.260771 1482 raft_consensus.cc:729] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d3f03e4888498db89cfd644bcf693d, State: Initialized, Role: FOLLOWER
I20260809 22:59:38.261088 1482 consensus_queue.cc:260] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.262475 1482 ts_tablet_manager.cc:1428] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:38.373297 1389 raft_consensus.cc:482] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:38.373519 1389 raft_consensus.cc:504] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.374269 1389 leader_election.cc:288] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483), 412c681c4ac04123989843e747b439ff (127.24.31.195:41963)
I20260809 22:59:38.375026 977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3d3b678a5044b8ac34d817f31e4352" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d3f03e4888498db89cfd644bcf693d" is_pre_election: true
I20260809 22:59:38.375149 1299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3d3b678a5044b8ac34d817f31e4352" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412c681c4ac04123989843e747b439ff" is_pre_election: true
I20260809 22:59:38.375404 1299 raft_consensus.cc:2454] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 0.
I20260809 22:59:38.375391 977 raft_consensus.cc:2454] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 0.
I20260809 22:59:38.376015 1063 leader_election.cc:302] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412c681c4ac04123989843e747b439ff, 87140d10272047d5a56cc9b5e6f28aad; no voters:
I20260809 22:59:38.376487 1389 raft_consensus.cc:2781] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:38.376605 1389 raft_consensus.cc:482] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:38.376693 1389 raft_consensus.cc:3037] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:38.379673 1389 raft_consensus.cc:504] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.380751 977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3d3b678a5044b8ac34d817f31e4352" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d3f03e4888498db89cfd644bcf693d"
I20260809 22:59:38.380929 1299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f3d3b678a5044b8ac34d817f31e4352" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412c681c4ac04123989843e747b439ff"
I20260809 22:59:38.381145 1299 raft_consensus.cc:3037] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:38.381280 977 raft_consensus.cc:3037] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:38.384183 1299 raft_consensus.cc:2454] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 1.
I20260809 22:59:38.384569 1063 leader_election.cc:302] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412c681c4ac04123989843e747b439ff, 87140d10272047d5a56cc9b5e6f28aad; no voters:
I20260809 22:59:38.384909 977 raft_consensus.cc:2454] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 1.
I20260809 22:59:38.385681 1389 leader_election.cc:288] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 election: Requested vote from peers 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483), 412c681c4ac04123989843e747b439ff (127.24.31.195:41963)
I20260809 22:59:38.385893 1389 raft_consensus.cc:2781] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:38.386032 1389 raft_consensus.cc:686] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 1 LEADER]: Becoming Leader. State: Replica: 87140d10272047d5a56cc9b5e6f28aad, State: Running, Role: LEADER
I20260809 22:59:38.386301 1389 consensus_queue.cc:237] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.389185 805 catalog_manager.cc:5495] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad reported cstate change: term changed from 0 to 1, leader changed from <none> to 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194). New cstate: current_term: 1 leader_uuid: "87140d10272047d5a56cc9b5e6f28aad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } health_report { overall_health: UNKNOWN } } }
I20260809 22:59:38.404956 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:38.399118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53024 (local address 127.24.31.194:34911)
0809 22:59:38.399679 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:38.399686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:38.399707 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:59:38.399775 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:38.399781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:38.399874 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:38.399996 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:38.400362 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.401556 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.402847 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.403596 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.403740 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.404165 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:38.404307 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:38.404424 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:38.404542 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 22:59:38.404628 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":279,"threads_started":1}
I20260809 22:59:38.407918 1299 raft_consensus.cc:1261] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Refusing update from remote peer 87140d10272047d5a56cc9b5e6f28aad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:38.407990 977 raft_consensus.cc:1261] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Refusing update from remote peer 87140d10272047d5a56cc9b5e6f28aad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:38.408636 1389 consensus_queue.cc:1035] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:38.408905 1486 consensus_queue.cc:1035] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [LEADER]: Connected to new peer: Peer: permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:38.428828 805 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:38.429105 805 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:38.432273 805 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e: Sending DeleteTablet for 3 replicas of tablet 1f3d3b678a5044b8ac34d817f31e4352
I20260809 22:59:38.433105 957 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f3d3b678a5044b8ac34d817f31e4352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:38 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:42132
I20260809 22:59:38.433123 1116 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f3d3b678a5044b8ac34d817f31e4352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:38 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:53010
I20260809 22:59:38.433454 1461 tablet_replica.cc:330] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: stopping tablet replica
I20260809 22:59:38.433454 1462 tablet_replica.cc:330] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: stopping tablet replica
I20260809 22:59:38.433415 1272 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f3d3b678a5044b8ac34d817f31e4352 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:38 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:47802
I20260809 22:59:38.433650 1462 raft_consensus.cc:2229] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:59:38.433650 1461 raft_consensus.cc:2229] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:38.433732 1463 tablet_replica.cc:330] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: stopping tablet replica
I20260809 22:59:38.433864 1461 raft_consensus.cc:2258] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:38.433876 1463 raft_consensus.cc:2229] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:38.433943 1462 raft_consensus.cc:2258] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:38.434077 1463 raft_consensus.cc:2258] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:38.435247 1462 ts_tablet_manager.cc:1905] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:38.435279 1463 ts_tablet_manager.cc:1905] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:38.439880 1462 ts_tablet_manager.cc:1918] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:38.440052 1462 log.cc:1196] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/1f3d3b678a5044b8ac34d817f31e4352
I20260809 22:59:38.440464 1462 ts_tablet_manager.cc:1939] T 1f3d3b678a5044b8ac34d817f31e4352 P 87140d10272047d5a56cc9b5e6f28aad: Deleting consensus metadata
I20260809 22:59:38.440692 1461 ts_tablet_manager.cc:1905] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:38.441541 790 catalog_manager.cc:4841] TS 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194:34911): tablet 1f3d3b678a5044b8ac34d817f31e4352 (table test-table [id=5ecdd5faa75e4d51b84ea455ea047680]) successfully deleted
I20260809 22:59:38.443383 805 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 22:59:38.443982 1463 ts_tablet_manager.cc:1918] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260809 22:59:38.444159 805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 22:59:38.444207 1463 log.cc:1196] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/1f3d3b678a5044b8ac34d817f31e4352
I20260809 22:59:38.444787 1463 ts_tablet_manager.cc:1939] T 1f3d3b678a5044b8ac34d817f31e4352 P 412c681c4ac04123989843e747b439ff: Deleting consensus metadata
I20260809 22:59:38.446072 788 catalog_manager.cc:4841] TS 412c681c4ac04123989843e747b439ff (127.24.31.195:41963): tablet 1f3d3b678a5044b8ac34d817f31e4352 (table test-table [id=5ecdd5faa75e4d51b84ea455ea047680]) successfully deleted
I20260809 22:59:38.447762 1461 ts_tablet_manager.cc:1918] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 22:59:38.447916 1461 log.cc:1196] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/1f3d3b678a5044b8ac34d817f31e4352
I20260809 22:59:38.448573 1461 ts_tablet_manager.cc:1939] T 1f3d3b678a5044b8ac34d817f31e4352 P 85d3f03e4888498db89cfd644bcf693d: Deleting consensus metadata
I20260809 22:59:38.449672 790 catalog_manager.cc:4841] TS 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483): tablet 1f3d3b678a5044b8ac34d817f31e4352 (table test-table [id=5ecdd5faa75e4d51b84ea455ea047680]) successfully deleted
I20260809 22:59:38.451802 957 tablet_service.cc:1467] Processing CreateTablet for tablet 4ccb7aae3ece4765aacb07a20cbbedcf (DEFAULT_TABLE table=test-table [id=5879adf069814df5929a0de829dac92c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:38.452241 957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ccb7aae3ece4765aacb07a20cbbedcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:38.452353 1272 tablet_service.cc:1467] Processing CreateTablet for tablet 4ccb7aae3ece4765aacb07a20cbbedcf (DEFAULT_TABLE table=test-table [id=5879adf069814df5929a0de829dac92c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:38.452365 1116 tablet_service.cc:1467] Processing CreateTablet for tablet 4ccb7aae3ece4765aacb07a20cbbedcf (DEFAULT_TABLE table=test-table [id=5879adf069814df5929a0de829dac92c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 22:59:38.452754 1116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ccb7aae3ece4765aacb07a20cbbedcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:38.452744 1272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ccb7aae3ece4765aacb07a20cbbedcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 22:59:38.458531 1482 tablet_bootstrap.cc:492] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: Bootstrap starting.
I20260809 22:59:38.460291 1482 tablet_bootstrap.cc:654] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:38.461165 1478 tablet_bootstrap.cc:492] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: Bootstrap starting.
I20260809 22:59:38.462905 1478 tablet_bootstrap.cc:654] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:38.463826 1480 tablet_bootstrap.cc:492] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: Bootstrap starting.
I20260809 22:59:38.465337 1480 tablet_bootstrap.cc:654] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:38.467427 1482 tablet_bootstrap.cc:492] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: No bootstrap required, opened a new log
I20260809 22:59:38.467603 1482 ts_tablet_manager.cc:1397] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260809 22:59:38.468757 1482 raft_consensus.cc:348] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.469043 1482 raft_consensus.cc:374] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:38.469132 1482 raft_consensus.cc:729] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85d3f03e4888498db89cfd644bcf693d, State: Initialized, Role: FOLLOWER
I20260809 22:59:38.469419 1482 consensus_queue.cc:260] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.472415 1482 ts_tablet_manager.cc:1428] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260809 22:59:38.475994 1478 tablet_bootstrap.cc:492] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: No bootstrap required, opened a new log
I20260809 22:59:38.476186 1478 ts_tablet_manager.cc:1397] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260809 22:59:38.477025 1478 raft_consensus.cc:348] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.477341 1478 raft_consensus.cc:374] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:38.477437 1478 raft_consensus.cc:729] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87140d10272047d5a56cc9b5e6f28aad, State: Initialized, Role: FOLLOWER
I20260809 22:59:38.477675 1480 tablet_bootstrap.cc:492] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: No bootstrap required, opened a new log
I20260809 22:59:38.477741 1478 consensus_queue.cc:260] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.477890 1480 ts_tablet_manager.cc:1397] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260809 22:59:38.478317 1478 ts_tablet_manager.cc:1428] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 22:59:38.478678 1480 raft_consensus.cc:348] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.478988 1480 raft_consensus.cc:374] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:38.479102 1480 raft_consensus.cc:729] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412c681c4ac04123989843e747b439ff, State: Initialized, Role: FOLLOWER
I20260809 22:59:38.479362 1480 consensus_queue.cc:260] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.480685 1480 ts_tablet_manager.cc:1428] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 22:59:38.641486 1494 raft_consensus.cc:482] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 22:59:38.641714 1494 raft_consensus.cc:504] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.642676 1494 leader_election.cc:288] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483), 412c681c4ac04123989843e747b439ff (127.24.31.195:41963)
I20260809 22:59:38.643482 977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ccb7aae3ece4765aacb07a20cbbedcf" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d3f03e4888498db89cfd644bcf693d" is_pre_election: true
I20260809 22:59:38.643505 1299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ccb7aae3ece4765aacb07a20cbbedcf" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412c681c4ac04123989843e747b439ff" is_pre_election: true
I20260809 22:59:38.643774 977 raft_consensus.cc:2454] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 0.
I20260809 22:59:38.643857 1299 raft_consensus.cc:2454] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 0.
I20260809 22:59:38.644346 1065 leader_election.cc:302] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 412c681c4ac04123989843e747b439ff, 85d3f03e4888498db89cfd644bcf693d, 87140d10272047d5a56cc9b5e6f28aad; no voters:
I20260809 22:59:38.644696 1494 raft_consensus.cc:2781] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 22:59:38.644829 1494 raft_consensus.cc:482] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 22:59:38.644936 1494 raft_consensus.cc:3037] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:38.647073 1494 raft_consensus.cc:504] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.647629 1494 leader_election.cc:288] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 election: Requested vote from peers 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483), 412c681c4ac04123989843e747b439ff (127.24.31.195:41963)
I20260809 22:59:38.648185 1299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ccb7aae3ece4765aacb07a20cbbedcf" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412c681c4ac04123989843e747b439ff"
I20260809 22:59:38.648183 977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ccb7aae3ece4765aacb07a20cbbedcf" candidate_uuid: "87140d10272047d5a56cc9b5e6f28aad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85d3f03e4888498db89cfd644bcf693d"
I20260809 22:59:38.648393 1299 raft_consensus.cc:3037] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:38.648407 977 raft_consensus.cc:3037] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:38.650597 1299 raft_consensus.cc:2454] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 1.
I20260809 22:59:38.650766 977 raft_consensus.cc:2454] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87140d10272047d5a56cc9b5e6f28aad in term 1.
I20260809 22:59:38.651091 1063 leader_election.cc:302] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412c681c4ac04123989843e747b439ff, 87140d10272047d5a56cc9b5e6f28aad; no voters:
I20260809 22:59:38.651355 1494 raft_consensus.cc:2781] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:38.651597 1494 raft_consensus.cc:686] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 1 LEADER]: Becoming Leader. State: Replica: 87140d10272047d5a56cc9b5e6f28aad, State: Running, Role: LEADER
I20260809 22:59:38.651916 1494 consensus_queue.cc:237] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } }
I20260809 22:59:38.654670 804 catalog_manager.cc:5495] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad reported cstate change: term changed from 0 to 1, leader changed from <none> to 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194). New cstate: current_term: 1 leader_uuid: "87140d10272047d5a56cc9b5e6f28aad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87140d10272047d5a56cc9b5e6f28aad" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 34911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 } health_report { overall_health: UNKNOWN } } }
I20260809 22:59:38.725102 1505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:38.719811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:42483 (local address 127.24.31.194:38373)
0809 22:59:38.720406 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:38.720423 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:38.720531 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:38.720842 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:38.720845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:38.720855 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:59:38.721205 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:38.721222 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.722197 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.722201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:38.723269 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:38.723282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.724089 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.724093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:38.724213 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.724266 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:59:38.724627 (+ 361us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:59:38.724636 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 22:59:38.724694 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 22:59:38.724802 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":320,"threads_started":1}
I20260809 22:59:38.725104 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:38.719952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:38373 (local address 127.24.31.193:42483)
0809 22:59:38.720465 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:38.720470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:38.720488 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:59:38.720653 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:38.720656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:38.720722 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:38.720813 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:38.721359 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.722073 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.723430 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.723918 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.724041 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.724431 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:38.724539 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:38.724615 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:38.724744 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 22:59:38.724815 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":263,"threads_started":1}
I20260809 22:59:38.733413 1299 raft_consensus.cc:1261] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Refusing update from remote peer 87140d10272047d5a56cc9b5e6f28aad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:38.733587 977 raft_consensus.cc:1261] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Refusing update from remote peer 87140d10272047d5a56cc9b5e6f28aad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 22:59:38.734030 1389 consensus_queue.cc:1035] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [LEADER]: Connected to new peer: Peer: permanent_uuid: "412c681c4ac04123989843e747b439ff" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:38.734292 1494 consensus_queue.cc:1035] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [LEADER]: Connected to new peer: Peer: permanent_uuid: "85d3f03e4888498db89cfd644bcf693d" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 42483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 22:59:38.738179 1439 mvcc.cc:205] Tried to move back new op lower bound from 7316751887287336960 to 7316751886960177152. Current Snapshot: MvccSnapshot[applied={T|T < 7316751887287336960}]
I20260809 22:59:38.759516 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:38.754268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:48399 (local address 127.24.31.194:34911)
0809 22:59:38.754474 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:38.754482 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:38.754743 (+ 261us) server_negotiation.cc:407] Connection header received
0809 22:59:38.754962 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:38.754967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:38.755042 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:38.755137 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 22:59:38.755605 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.756458 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.758004 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.758405 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.758501 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.758835 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 22:59:38.758930 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 22:59:38.758996 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:38.759146 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 22:59:38.759226 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:59:38.759593 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:38.754066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:34911 (local address 127.24.31.193:48399)
0809 22:59:38.754648 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:38.754663 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:38.754813 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:38.755152 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:38.755157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:38.755169 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 22:59:38.755405 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:38.755414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.756574 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.756579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:38.757875 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:38.757888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.758547 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.758551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:38.758666 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.758701 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 22:59:38.759013 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 22:59:38.759021 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 22:59:38.759112 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 22:59:38.759244 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":297,"threads_started":1}
I20260809 22:59:38.785918 805 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:38.786188 805 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 22:59:38.788937 805 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cb84b66bbf245e3a50b36ab42b2035e: Sending DeleteTablet for 3 replicas of tablet 4ccb7aae3ece4765aacb07a20cbbedcf
I20260809 22:59:38.789775 957 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ccb7aae3ece4765aacb07a20cbbedcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:38 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:42132
I20260809 22:59:38.790020 1272 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ccb7aae3ece4765aacb07a20cbbedcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:38 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:47802
I20260809 22:59:38.789958 1116 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ccb7aae3ece4765aacb07a20cbbedcf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 22:59:38 UTC) from {username='oryx', principal='oryx/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:53010
I20260809 22:59:38.790318 1461 tablet_replica.cc:330] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: stopping tablet replica
I20260809 22:59:38.790447 1462 tablet_replica.cc:330] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: stopping tablet replica
I20260809 22:59:38.790488 1461 raft_consensus.cc:2229] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:38.790647 1461 raft_consensus.cc:2258] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:38.790628 1462 raft_consensus.cc:2229] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 1 LEADER]: Raft consensus shutting down.
I20260809 22:59:38.790982 1462 raft_consensus.cc:2258] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:38.791113 1463 tablet_replica.cc:330] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: stopping tablet replica
I20260809 22:59:38.791250 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 882
I20260809 22:59:38.791322 1463 raft_consensus.cc:2229] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 22:59:38.791555 1463 raft_consensus.cc:2258] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 22:59:38.792649 1463 ts_tablet_manager.cc:1905] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:38.792718 1461 ts_tablet_manager.cc:1905] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:38.793960 1462 ts_tablet_manager.cc:1905] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 22:59:38.799269 1463 ts_tablet_manager.cc:1918] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:59:38.799530 1463 log.cc:1196] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/4ccb7aae3ece4765aacb07a20cbbedcf
I20260809 22:59:38.799651 1461 ts_tablet_manager.cc:1918] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:59:38.799793 1461 log.cc:1196] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/4ccb7aae3ece4765aacb07a20cbbedcf
I20260809 22:59:38.800137 1463 ts_tablet_manager.cc:1939] T 4ccb7aae3ece4765aacb07a20cbbedcf P 412c681c4ac04123989843e747b439ff: Deleting consensus metadata
I20260809 22:59:38.800259 1461 ts_tablet_manager.cc:1939] T 4ccb7aae3ece4765aacb07a20cbbedcf P 85d3f03e4888498db89cfd644bcf693d: Deleting consensus metadata
I20260809 22:59:38.801337 788 catalog_manager.cc:4841] TS 412c681c4ac04123989843e747b439ff (127.24.31.195:41963): tablet 4ccb7aae3ece4765aacb07a20cbbedcf (table test-table [id=5879adf069814df5929a0de829dac92c]) successfully deleted
I20260809 22:59:38.801548 790 catalog_manager.cc:4841] TS 85d3f03e4888498db89cfd644bcf693d (127.24.31.193:42483): tablet 4ccb7aae3ece4765aacb07a20cbbedcf (table test-table [id=5879adf069814df5929a0de829dac92c]) successfully deleted
Aug 09 22:59:38 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.193@KRBTEST.COM
I20260809 22:59:38.811906 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:38.792618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42150 (local address 127.24.31.193:42483)
0809 22:59:38.792817 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:38.792823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:38.792906 (+ 83us) server_negotiation.cc:407] Connection header received
0809 22:59:38.793205 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:38.793210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:38.793290 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:38.793367 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:38.795138 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.796209 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.798176 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.798872 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.798965 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.804271 (+ 5306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:38.804293 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:38.804315 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:38.804366 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:38.808471 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:38.809142 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:38.809181 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:38.809185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:38.809374 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:38.809888 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:38.809894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:38.809898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:38.810401 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:38.810697 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:38.811243 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 22:59:38.811511 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 22:59:38.812381 1462 ts_tablet_manager.cc:1918] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 22:59:38.812583 1462 log.cc:1196] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/4ccb7aae3ece4765aacb07a20cbbedcf
I20260809 22:59:38.812975 1033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:38.813220 1462 ts_tablet_manager.cc:1939] T 4ccb7aae3ece4765aacb07a20cbbedcf P 87140d10272047d5a56cc9b5e6f28aad: Deleting consensus metadata
I20260809 22:59:38.815053 790 catalog_manager.cc:4841] TS 87140d10272047d5a56cc9b5e6f28aad (127.24.31.194:34911): tablet 4ccb7aae3ece4765aacb07a20cbbedcf (table test-table [id=5879adf069814df5929a0de829dac92c]) successfully deleted
I20260809 22:59:38.940971 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 882
I20260809 22:59:38.967723 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 1042
Aug 09 22:59:38 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.194@KRBTEST.COM
I20260809 22:59:38.983311 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:38.968582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.24.31.194:34911)
0809 22:59:38.968758 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:38.968762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:38.968780 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:59:38.968968 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:38.968972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:38.969035 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:38.969127 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:38.970455 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.971180 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.972370 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:38.972804 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:38.972890 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:38.977308 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:38.977323 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:38.977339 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:38.977373 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:38.980375 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:38.981050 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:38.981063 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:38.981067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:38.981334 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:38.981704 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:38.981708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:38.981710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:38.982081 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:38.982315 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:38.982655 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 22:59:38.982834 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 22:59:38.984310 1191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:39.114437 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 1042
I20260809 22:59:39.141041 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 1201
Aug 09 22:59:39 dist-test-slave-vxj2 krb5kdc[750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.31.195@KRBTEST.COM
I20260809 22:59:39.156617 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:39.141915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47822 (local address 127.24.31.195:41963)
0809 22:59:39.142842 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:39.142847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:39.142862 (+ 15us) server_negotiation.cc:407] Connection header received
0809 22:59:39.142915 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:39.142919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:39.142976 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:39.143073 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:39.144364 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:39.145114 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:39.146390 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:39.146779 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:39.146871 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:39.151254 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:39.151269 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:39.151286 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:39.151321 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:39.154126 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:39.154746 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:39.154753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:39.154755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:39.154878 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:39.155220 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:39.155223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:39.155225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:39.155572 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:39.155785 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:39.156066 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 22:59:39.156235 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":683,"threads_started":1}
I20260809 22:59:39.157509 1345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:39.285590 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 1201
I20260809 22:59:39.315296 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 766
I20260809 22:59:39.316468 849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:39.440158 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 766
2026-08-09T22:59:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7214 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:59:39 dist-test-slave-vxj2 krb5kdc[1520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:59:39 dist-test-slave-vxj2 krb5kdc[1520](info): set up 2 sockets
Aug 09 22:59:39 dist-test-slave-vxj2 krb5kdc[1520](info): commencing operation
krb5kdc: starting...
W20260809 22:59:40.632296 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.007s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:59:40 dist-test-slave-vxj2 krb5kdc[1520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:59:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:59:40.876950 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42379
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:42379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:41.205601 1535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:41.205936 1535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:41.206095 1535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:41.213385 1535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:41.213536 1535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:41.213634 1535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:41.213737 1535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:41.213829 1535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:41.223588 1535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:42379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:41.224824 1535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:41.226159 1535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:41.238131 1543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:41.238176 1541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:41.238207 1540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:41.239003 1535 server_base.cc:1034] running on GCE node
I20260809 22:59:41.239746 1535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:41.241134 1535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:41.242419 1535 hybrid_clock.cc:648] HybridClock initialized: now 1786316381242364 us; error 46 us; skew 500 ppm
I20260809 22:59:41.242815 1535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T22:59:41Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1872 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 22:59:41 dist-test-slave-vxj2 krb5kdc[1549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 22:59:41 dist-test-slave-vxj2 krb5kdc[1549](info): set up 2 sockets
Aug 09 22:59:41 dist-test-slave-vxj2 krb5kdc[1549](info): commencing operation
krb5kdc: starting...
W20260809 22:59:42.522871 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 22:59:42 dist-test-slave-vxj2 krb5kdc[1549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T22:59:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T22:59:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 22:59:42.757354 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:45247
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:45247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:43.090024 1562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:43.090292 1562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:43.090478 1562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:43.097488 1562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:43.097579 1562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:43.097625 1562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:43.097750 1562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:43.097821 1562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:43.107504 1562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:45247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:45247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:43.108842 1562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:43.110148 1562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:43.121876 1575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:43.121834 1570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:43.121987 1569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:43.123049 1562 server_base.cc:1034] running on GCE node
I20260809 22:59:43.123824 1562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:43.125093 1562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:43.126351 1562 hybrid_clock.cc:648] HybridClock initialized: now 1786316383126310 us; error 25 us; skew 500 ppm
I20260809 22:59:43.126698 1562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:43 dist-test-slave-vxj2 krb5kdc[1549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316383, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:43.134207 1562 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:59:43.135736 1562 webserver.cc:460] Webserver started at http://127.24.31.254:43013/ using document root <none> and password file <none>
I20260809 22:59:43.136411 1562 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:43.136555 1562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:43.136870 1562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:43.139827 1562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "1d23b58384b64fd58dfd2477c406c587"
format_stamp: "Formatted at 2026-08-09 22:59:43 on dist-test-slave-vxj2"
server_key: "7a45f6fdc81e928e2d46b8da42f64f87"
server_key_iv: "0e0e62503e442227a7a5f06ca56878df"
server_key_version: "encryptionkey@0"
I20260809 22:59:43.140565 1562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "1d23b58384b64fd58dfd2477c406c587"
format_stamp: "Formatted at 2026-08-09 22:59:43 on dist-test-slave-vxj2"
server_key: "7a45f6fdc81e928e2d46b8da42f64f87"
server_key_iv: "0e0e62503e442227a7a5f06ca56878df"
server_key_version: "encryptionkey@0"
I20260809 22:59:43.145833 1562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 22:59:43.149087 1583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:43.150465 1562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:43.150637 1562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "1d23b58384b64fd58dfd2477c406c587"
format_stamp: "Formatted at 2026-08-09 22:59:43 on dist-test-slave-vxj2"
server_key: "7a45f6fdc81e928e2d46b8da42f64f87"
server_key_iv: "0e0e62503e442227a7a5f06ca56878df"
server_key_version: "encryptionkey@0"
I20260809 22:59:43.150831 1562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:43.172508 1562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:43.176859 1562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:43.177172 1562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:43.195502 1562 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:45247
I20260809 22:59:43.195503 1640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:45247 every 8 connection(s)
I20260809 22:59:43.196909 1562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:43.199527 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 1562
I20260809 22:59:43.199759 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 22:59:43.200493 24703 external_mini_cluster.cc:1428] Setting key 506fdcd7e234b8a4076c92f068dc65ad
I20260809 22:59:43.201570 1642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 22:59:43 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:43.215991 1642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: Bootstrap starting.
I20260809 22:59:43.219354 1642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: Neither blocks nor log segments found. Creating new log.
I20260809 22:59:43.220357 1642 log.cc:824] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:43.223116 1645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:43.202470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.24.31.254:45247)
0809 22:59:43.203272 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:43.203283 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:43.203323 (+ 40us) server_negotiation.cc:407] Connection header received
0809 22:59:43.204888 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:43.204908 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:43.205353 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:43.205687 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:43.207647 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:43.209000 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:43.210780 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:43.211395 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:43.211540 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:43.216058 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:43.216065 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:43.216078 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:43.216142 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:43.219141 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:43.219871 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:43.219878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:43.219884 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:43.220043 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:43.220432 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:43.220437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:43.220440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:43.221005 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:43.221359 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:43.221788 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 22:59:43.222189 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":393,"threads_started":1}
I20260809 22:59:43.223234 1642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: No bootstrap required, opened a new log
I20260809 22:59:43.231247 1642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } }
I20260809 22:59:43.231684 1642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 22:59:43.231804 1642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d23b58384b64fd58dfd2477c406c587, State: Initialized, Role: FOLLOWER
I20260809 22:59:43.232338 1642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } }
I20260809 22:59:43.232565 1642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:43.232725 1642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:43.232901 1642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 0 FOLLOWER]: Advancing to term 1
I20260809 22:59:43.235070 1642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } }
I20260809 22:59:43.235599 1642 leader_election.cc:302] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d23b58384b64fd58dfd2477c406c587; no voters:
I20260809 22:59:43.236186 1642 leader_election.cc:288] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 22:59:43.236320 1647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 22:59:43.237560 1647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 1 LEADER]: Becoming Leader. State: Replica: 1d23b58384b64fd58dfd2477c406c587, State: Running, Role: LEADER
I20260809 22:59:43.238125 1647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } }
I20260809 22:59:43.238368 1642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:43.241067 1650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d23b58384b64fd58dfd2477c406c587. Latest consensus state: current_term: 1 leader_uuid: "1d23b58384b64fd58dfd2477c406c587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } } }
I20260809 22:59:43.241254 1648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d23b58384b64fd58dfd2477c406c587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } } }
I20260809 22:59:43.241516 1650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:43.241532 1648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:43.243242 1657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:43.247642 1657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:43.256824 1657 catalog_manager.cc:1347] Generated new cluster ID: ddd12c69f4ee483d81ce578472fd2c8a
I20260809 22:59:43.256960 1657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:43.283641 1657 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 22:59:43.285555 1657 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:43.297930 1657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: Generated new TSK 0
I20260809 22:59:43.298918 1657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 22:59:43.365135 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:45247
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:43.701226 1670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:43.701543 1670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:43.701705 1670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:43.708774 1670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:43.708925 1670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:43.709188 1670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:43.718608 1670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:45247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:43.719899 1670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:43.721256 1670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:43.734071 1675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:43.734114 1678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:43.734161 1676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:43.735918 1670 server_base.cc:1034] running on GCE node
I20260809 22:59:43.736505 1670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:43.737413 1670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:43.738639 1670 hybrid_clock.cc:648] HybridClock initialized: now 1786316383738604 us; error 50 us; skew 500 ppm
I20260809 22:59:43.738981 1670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:43 dist-test-slave-vxj2 krb5kdc[1549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316383, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:43.742405 1670 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 22:59:43.743845 1670 webserver.cc:460] Webserver started at http://127.24.31.193:35567/ using document root <none> and password file <none>
I20260809 22:59:43.744550 1670 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:43.744699 1670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:43.745009 1670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:43.747965 1670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "a92ff34637404f44bcd8afad4c60f29c"
format_stamp: "Formatted at 2026-08-09 22:59:43 on dist-test-slave-vxj2"
server_key: "1cb911482b409023149dabe010594e68"
server_key_iv: "c4fd0c988a41ff1d4ae04a5736c52534"
server_key_version: "encryptionkey@0"
I20260809 22:59:43.748814 1670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "a92ff34637404f44bcd8afad4c60f29c"
format_stamp: "Formatted at 2026-08-09 22:59:43 on dist-test-slave-vxj2"
server_key: "1cb911482b409023149dabe010594e68"
server_key_iv: "c4fd0c988a41ff1d4ae04a5736c52534"
server_key_version: "encryptionkey@0"
I20260809 22:59:43.753997 1670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 22:59:43.757376 1686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:43.758661 1670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:59:43.758841 1670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "a92ff34637404f44bcd8afad4c60f29c"
format_stamp: "Formatted at 2026-08-09 22:59:43 on dist-test-slave-vxj2"
server_key: "1cb911482b409023149dabe010594e68"
server_key_iv: "c4fd0c988a41ff1d4ae04a5736c52534"
server_key_version: "encryptionkey@0"
I20260809 22:59:43.759043 1670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:43.773975 1670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:43.778239 1670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:43.778537 1670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:43.779484 1670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:43.780736 1670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:43.780841 1670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:43.780993 1670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:43.781071 1670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:43.815260 1670 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:34085
I20260809 22:59:43.815303 1811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:34085 every 8 connection(s)
I20260809 22:59:43.816569 1670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:43.821142 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 1670
I20260809 22:59:43.821384 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 22:59:43.822196 24703 external_mini_cluster.cc:1428] Setting key 36933b62016aba093eb781ca3a736442
Aug 09 22:59:43 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316383, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:43.839126 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:43.819145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:57981 (local address 127.24.31.254:45247)
0809 22:59:43.819622 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:43.819628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:43.820660 (+ 1032us) server_negotiation.cc:407] Connection header received
0809 22:59:43.823245 (+ 2585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:43.823250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:43.823351 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:43.823487 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:43.825331 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:43.826052 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:43.828490 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:43.828869 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:43.828966 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:43.833290 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:43.833295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:43.833298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:43.833332 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:43.835916 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:43.836642 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:43.836647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:43.836657 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:43.836773 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:43.837235 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:43.837239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:43.837243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:43.837511 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:43.837658 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:43.838406 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 22:59:43.838650 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":217,"threads_started":1}
I20260809 22:59:43.840075 1815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:43.819740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:57981)
0809 22:59:43.820479 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:43.820517 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:43.822987 (+ 2470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:43.823713 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:43.823722 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:43.824328 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:43.825123 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:43.825183 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:43.826224 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:43.826229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:43.828343 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:43.828360 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:43.829097 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:43.829106 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:43.829453 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:43.830184 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:43.830217 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:43.833026 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:43.836060 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:43.836069 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:43.836085 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:43.836512 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:43.836885 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:43.836890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:43.836894 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:43.837096 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:43.837648 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:43.837656 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:43.838014 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 22:59:43.838289 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 22:59:43.838805 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":311,"threads_started":1}
I20260809 22:59:43.841717 1812 heartbeater.cc:347] Connected to a master server at 127.24.31.254:45247
I20260809 22:59:43.842056 1812 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:43.842928 1812 heartbeater.cc:511] Master 127.24.31.254:45247 requested a full tablet report, sending...
I20260809 22:59:43.845507 1604 ts_manager.cc:194] Registered new tserver with Master: a92ff34637404f44bcd8afad4c60f29c (127.24.31.193:34085)
I20260809 22:59:43.847767 1604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:57981
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 22:59:43.878334 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:45247
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:44.210394 1819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:44.210660 1819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:44.210770 1819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:44.218083 1819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:44.218200 1819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:44.218330 1819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:44.228085 1819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:45247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:44.229435 1819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:44.230844 1819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:44.244778 1827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:44.244918 1826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:44.245519 1819 server_base.cc:1034] running on GCE node
W20260809 22:59:44.245705 1829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:44.246434 1819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:44.247486 1819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:44.248754 1819 hybrid_clock.cc:648] HybridClock initialized: now 1786316384248684 us; error 46 us; skew 500 ppm
I20260809 22:59:44.249267 1819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:44 dist-test-slave-vxj2 krb5kdc[1549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:44.252490 1819 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 22:59:44.254102 1819 webserver.cc:460] Webserver started at http://127.24.31.194:38667/ using document root <none> and password file <none>
I20260809 22:59:44.254837 1819 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:44.254947 1819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:44.255211 1819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:44.258320 1819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "118c422e658d4e07909b0279bb4956e9"
format_stamp: "Formatted at 2026-08-09 22:59:44 on dist-test-slave-vxj2"
server_key: "7d1a9575f0d48420cc225832732aa35b"
server_key_iv: "b132ca9e194356836e1ace7e309e6f21"
server_key_version: "encryptionkey@0"
I20260809 22:59:44.259166 1819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "118c422e658d4e07909b0279bb4956e9"
format_stamp: "Formatted at 2026-08-09 22:59:44 on dist-test-slave-vxj2"
server_key: "7d1a9575f0d48420cc225832732aa35b"
server_key_iv: "b132ca9e194356836e1ace7e309e6f21"
server_key_version: "encryptionkey@0"
I20260809 22:59:44.264628 1819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:59:44.268488 1837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:44.269804 1819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:44.270001 1819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "118c422e658d4e07909b0279bb4956e9"
format_stamp: "Formatted at 2026-08-09 22:59:44 on dist-test-slave-vxj2"
server_key: "7d1a9575f0d48420cc225832732aa35b"
server_key_iv: "b132ca9e194356836e1ace7e309e6f21"
server_key_version: "encryptionkey@0"
I20260809 22:59:44.270228 1819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:44.319492 1819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:44.323637 1819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:44.323932 1819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:44.324841 1819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:44.326207 1819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:44.326321 1819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:44.326479 1819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:44.326555 1819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:44.360558 1819 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:37105
I20260809 22:59:44.360615 1960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:37105 every 8 connection(s)
I20260809 22:59:44.361945 1819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:44.371534 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 1819
I20260809 22:59:44.371729 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 22:59:44.372277 24703 external_mini_cluster.cc:1428] Setting key 5730bf5fdafeae0ae608721859008971
Aug 09 22:59:44 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:44.383160 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:44.364581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:48117 (local address 127.24.31.254:45247)
0809 22:59:44.365041 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:44.365045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:44.365951 (+ 906us) server_negotiation.cc:407] Connection header received
0809 22:59:44.367755 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:44.367760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:44.367854 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:44.367979 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:44.369507 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:44.370241 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:44.372150 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:44.372660 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:44.372776 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:44.376799 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:44.376813 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:44.376817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:44.376853 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:44.379675 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:44.380572 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:44.380577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:44.380580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:44.380719 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:44.381239 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:44.381244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:44.381246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:44.381544 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:44.381718 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:44.382472 (+ 754us) server_negotiation.cc:299] Negotiation successful
0809 22:59:44.382712 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":250,"threads_started":1}
I20260809 22:59:44.384178 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:44.365044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:48117)
0809 22:59:44.365777 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:44.365820 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:44.367495 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:44.368219 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:44.368228 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:44.368717 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:44.369332 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:44.369351 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:44.370407 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:44.370411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:44.372000 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:44.372016 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:44.372898 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:44.372904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:44.373149 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:44.373953 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:44.373988 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:44.376554 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:44.379828 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:44.379838 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:44.379854 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:44.380410 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:44.380843 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:44.380850 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:44.380854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:44.381068 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:44.381686 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:44.381695 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:44.382068 (+ 373us) client_negotiation.cc:769] Sending connection context
0809 22:59:44.382373 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 22:59:44.382931 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":275,"threads_started":1}
I20260809 22:59:44.385800 1961 heartbeater.cc:347] Connected to a master server at 127.24.31.254:45247
I20260809 22:59:44.386137 1961 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:44.387009 1961 heartbeater.cc:511] Master 127.24.31.254:45247 requested a full tablet report, sending...
I20260809 22:59:44.388732 1604 ts_manager.cc:194] Registered new tserver with Master: 118c422e658d4e07909b0279bb4956e9 (127.24.31.194:37105)
I20260809 22:59:44.390043 1604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:48117
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 22:59:44.428799 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:45247
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:44.756982 1967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:44.757282 1967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:44.757376 1967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:44.764580 1967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:44.764685 1967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:44.764804 1967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:44.774973 1967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:45247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:44.776201 1967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:44.777588 1967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:44.790788 1972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:44.790750 1975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:44.790848 1973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:44.794539 1967 server_base.cc:1034] running on GCE node
I20260809 22:59:44.795138 1967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:44.796095 1967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:44.797319 1967 hybrid_clock.cc:648] HybridClock initialized: now 1786316384797201 us; error 131 us; skew 500 ppm
I20260809 22:59:44.797639 1967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:44 dist-test-slave-vxj2 krb5kdc[1549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:44.800791 1967 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 22:59:44.802480 1967 webserver.cc:460] Webserver started at http://127.24.31.195:40561/ using document root <none> and password file <none>
I20260809 22:59:44.803313 1967 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:44.803459 1967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:44.803763 1967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 22:59:44.806646 1967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "cf096cea0f244fb49eeaa7d08bd7e46d"
format_stamp: "Formatted at 2026-08-09 22:59:44 on dist-test-slave-vxj2"
server_key: "5b6cd44fb780a6aa8e83017a5bdf6e0d"
server_key_iv: "0177b428566618679c650f5d9e0b0f52"
server_key_version: "encryptionkey@0"
I20260809 22:59:44.807442 1967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "cf096cea0f244fb49eeaa7d08bd7e46d"
format_stamp: "Formatted at 2026-08-09 22:59:44 on dist-test-slave-vxj2"
server_key: "5b6cd44fb780a6aa8e83017a5bdf6e0d"
server_key_iv: "0177b428566618679c650f5d9e0b0f52"
server_key_version: "encryptionkey@0"
I20260809 22:59:44.812549 1967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 22:59:44.816314 1983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:44.817905 1967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 22:59:44.818078 1967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "cf096cea0f244fb49eeaa7d08bd7e46d"
format_stamp: "Formatted at 2026-08-09 22:59:44 on dist-test-slave-vxj2"
server_key: "5b6cd44fb780a6aa8e83017a5bdf6e0d"
server_key_iv: "0177b428566618679c650f5d9e0b0f52"
server_key_version: "encryptionkey@0"
I20260809 22:59:44.818300 1967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:44.833813 1967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:44.837982 1967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:44.838212 1967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:44.839079 1967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:44.840391 1967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:44.840464 1967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:44.840678 1967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:44.840731 1967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:44.850901 1812 heartbeater.cc:503] Master 127.24.31.254:45247 was elected leader, sending a full tablet report...
I20260809 22:59:44.875516 1967 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:39565
I20260809 22:59:44.875566 2111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:39565 every 8 connection(s)
I20260809 22:59:44.876796 1967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:44.883909 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 1967
I20260809 22:59:44.884107 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 22:59:44.884688 24703 external_mini_cluster.cc:1428] Setting key 7146fe659daa8c80a4a92b5071f54427
Aug 09 22:59:44 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316384, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 22:59:44.897511 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:44.879509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:56951 (local address 127.24.31.254:45247)
0809 22:59:44.879691 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:44.879696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:44.880889 (+ 1193us) server_negotiation.cc:407] Connection header received
0809 22:59:44.882760 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:44.882764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:44.882835 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:44.882935 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:44.884890 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:44.885906 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:44.888036 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:44.888434 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:44.888532 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:44.891913 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:44.891918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:44.891921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:44.891965 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:44.894522 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:44.895245 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:44.895249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:44.895251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:44.895370 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:44.895996 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:44.895999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:44.896001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:44.896250 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:44.896379 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:44.897009 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 22:59:44.897251 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 22:59:44.898593 2114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:44.880020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:56951)
0809 22:59:44.880710 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:44.880753 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:44.882481 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:44.883127 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:44.883136 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:44.883859 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:44.884659 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:44.884688 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:44.886083 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:44.886090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:44.887880 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:44.887895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:44.888665 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:44.888671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:44.888899 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:44.889551 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:44.889574 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:44.891706 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 22:59:44.894685 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:44.894692 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:44.894703 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:44.895143 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:44.895611 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 22:59:44.895618 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 22:59:44.895624 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 22:59:44.895881 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 22:59:44.896359 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 22:59:44.896365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 22:59:44.896656 (+ 291us) client_negotiation.cc:769] Sending connection context
0809 22:59:44.896920 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 22:59:44.897365 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":265,"threads_started":1}
I20260809 22:59:44.900063 2112 heartbeater.cc:347] Connected to a master server at 127.24.31.254:45247
I20260809 22:59:44.900380 2112 heartbeater.cc:464] Registering TS with master...
I20260809 22:59:44.901144 2112 heartbeater.cc:511] Master 127.24.31.254:45247 requested a full tablet report, sending...
I20260809 22:59:44.902602 1604 ts_manager.cc:194] Registered new tserver with Master: cf096cea0f244fb49eeaa7d08bd7e46d (127.24.31.195:39565)
I20260809 22:59:44.903620 1604 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:56951
I20260809 22:59:44.912438 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab.
Entry for principal oryx/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab.
WARNING: no policy specified for oryx/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab.
Entry for principal oryx/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab.
WARNING: no policy specified for oryx/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab.
Entry for principal oryx/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab.
I20260809 22:59:45.058367 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 1670
Aug 09 22:59:45 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 22:59:45.075436 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:45.059634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54860 (local address 127.24.31.193:34085)
0809 22:59:45.060384 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:45.060392 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:45.060423 (+ 31us) server_negotiation.cc:407] Connection header received
0809 22:59:45.060495 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:45.060502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:45.060742 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:45.060896 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:45.062395 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:45.063604 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:45.064826 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:45.065523 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:45.065637 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:45.069887 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:45.069898 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:45.069912 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:45.069952 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:45.072569 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:45.073358 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:45.073364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:45.073367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:45.073488 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:45.073879 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:45.073882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:45.073884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:45.074218 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:45.074463 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:45.074822 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 22:59:45.075031 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":401,"threads_started":1}
I20260809 22:59:45.076843 1805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:45.163106 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 1670
I20260809 22:59:45.182416 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 1819
Aug 09 22:59:45 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 22:59:45.199424 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:45.183473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59768 (local address 127.24.31.194:37105)
0809 22:59:45.184028 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:45.184035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:45.184061 (+ 26us) server_negotiation.cc:407] Connection header received
0809 22:59:45.184128 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:45.184135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:45.184333 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:45.184467 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:45.185846 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:45.186895 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:45.188162 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:45.188695 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:45.188790 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:45.193092 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:45.193104 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:45.193118 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:45.193196 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:45.196561 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:45.197334 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:45.197343 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:45.197346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:45.197488 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:45.197837 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:45.197842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:45.197845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:45.198269 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:45.198538 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:45.198736 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 22:59:45.198998 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":294,"threads_started":1}
I20260809 22:59:45.200961 1956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:45.283973 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 1819
I20260809 22:59:45.302525 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 1967
Aug 09 22:59:45 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 22:59:45.319327 2127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:45.303546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53394 (local address 127.24.31.195:39565)
0809 22:59:45.304193 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:45.304200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:45.304229 (+ 29us) server_negotiation.cc:407] Connection header received
0809 22:59:45.304289 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:45.304295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:45.304504 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:45.304673 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:45.306181 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:45.307216 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:45.308477 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:45.309025 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:45.309205 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:45.313542 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:45.313553 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:45.313567 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:45.313599 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:45.316436 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:45.317099 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:45.317106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:45.317109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:45.317328 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:45.317710 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:45.317716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:45.317720 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:45.318072 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:45.318391 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:45.318686 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 22:59:45.318960 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":367,"threads_started":1}
I20260809 22:59:45.320528 2107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:45.406884 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 1967
I20260809 22:59:45.424566 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 1562
I20260809 22:59:45.426043 1636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 22:59:45.533383 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 1562
I20260809 22:59:45.557328 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:45247
--webserver_interface=127.24.31.254
--webserver_port=43013
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:45247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 22:59:45.886533 2132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:45.886873 2132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:45.887029 2132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:45.894294 2132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 22:59:45.894433 2132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:45.894548 2132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:45.894649 2132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 22:59:45.894737 2132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 22:59:45.904606 2132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:45247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:45247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=43013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:45.905997 2132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:45.907385 2132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:45.919508 2143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:45.919476 2140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:45.919552 2141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:45.922686 2132 server_base.cc:1034] running on GCE node
I20260809 22:59:45.923472 2132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:45.924801 2132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:45.926007 2132 hybrid_clock.cc:648] HybridClock initialized: now 1786316385925956 us; error 33 us; skew 500 ppm
I20260809 22:59:45.926385 2132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:45 dist-test-slave-vxj2 krb5kdc[1549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316385, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:45.929602 2132 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 22:59:45.931191 2132 webserver.cc:460] Webserver started at http://127.24.31.254:43013/ using document root <none> and password file <none>
I20260809 22:59:45.931860 2132 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:45.931977 2132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:45.937309 2132 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:45.940092 2150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:45.941483 2132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 22:59:45.941619 2132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "1d23b58384b64fd58dfd2477c406c587"
format_stamp: "Formatted at 2026-08-09 22:59:43 on dist-test-slave-vxj2"
server_key: "7a45f6fdc81e928e2d46b8da42f64f87"
server_key_iv: "0e0e62503e442227a7a5f06ca56878df"
server_key_version: "encryptionkey@0"
I20260809 22:59:45.942531 2132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:45.961776 2132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:45.965572 2132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:45.965790 2132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:45.985114 2132 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:45247
I20260809 22:59:45.985133 2209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:45247 every 8 connection(s)
I20260809 22:59:45.986752 2132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 22:59:45.991381 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 2132
I20260809 22:59:45.992379 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:34085
--local_ip_for_outbound_sockets=127.24.31.193
--tserver_master_addrs=127.24.31.254:45247
--webserver_port=35567
--webserver_interface=127.24.31.193
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.31.193
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 22:59:45.994513 2210 sys_catalog.cc:263] Verifying existing consensus state
I20260809 22:59:45.999615 2210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: Bootstrap starting.
I20260809 22:59:46.019958 2210 log.cc:824] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: Log is configured to *not* fsync() on all Append() calls
I20260809 22:59:46.029017 2210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 22:59:46.029611 2210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: Bootstrap complete.
I20260809 22:59:46.038861 2210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } }
I20260809 22:59:46.039685 2210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d23b58384b64fd58dfd2477c406c587, State: Initialized, Role: FOLLOWER
I20260809 22:59:46.040347 2210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } }
I20260809 22:59:46.040591 2210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 22:59:46.040717 2210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 22:59:46.040915 2210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 1 FOLLOWER]: Advancing to term 2
I20260809 22:59:46.044857 2210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } }
I20260809 22:59:46.045466 2210 leader_election.cc:302] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d23b58384b64fd58dfd2477c406c587; no voters:
I20260809 22:59:46.046096 2210 leader_election.cc:288] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 22:59:46.046316 2215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 22:59:46.048250 2210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 22:59:46.049450 2215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [term 2 LEADER]: Becoming Leader. State: Replica: 1d23b58384b64fd58dfd2477c406c587, State: Running, Role: LEADER
I20260809 22:59:46.050037 2215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } }
I20260809 22:59:46.052840 2216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1d23b58384b64fd58dfd2477c406c587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } } }
I20260809 22:59:46.052878 2217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d23b58384b64fd58dfd2477c406c587. Latest consensus state: current_term: 2 leader_uuid: "1d23b58384b64fd58dfd2477c406c587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d23b58384b64fd58dfd2477c406c587" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 45247 } } }
I20260809 22:59:46.053274 2216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:46.053345 2217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587 [sys.catalog]: This master's current role is: LEADER
I20260809 22:59:46.065861 2220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 22:59:46.077076 2220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 22:59:46.078572 2220 catalog_manager.cc:1259] Loaded cluster ID: ddd12c69f4ee483d81ce578472fd2c8a
I20260809 22:59:46.078729 2220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 22:59:46.082791 2220 catalog_manager.cc:1504] Loading token signing keys...
I20260809 22:59:46.084189 2220 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d23b58384b64fd58dfd2477c406c587: Loaded TSK: 0
I20260809 22:59:46.085757 2220 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 22:59:46.338259 2212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:46.338534 2212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:46.338673 2212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:46.346082 2212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:46.346208 2212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:46.346393 2212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 22:59:46.356509 2212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.193.keytab
--principal=oryx/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:34085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=35567
--tserver_master_addrs=127.24.31.254:45247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:46.357934 2212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:46.359310 2212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:46.372947 2238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:46.373229 2237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:46.373118 2243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:46.378548 2212 server_base.cc:1034] running on GCE node
I20260809 22:59:46.379179 2212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:46.380306 2212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:46.381496 2212 hybrid_clock.cc:648] HybridClock initialized: now 1786316386381445 us; error 58 us; skew 500 ppm
I20260809 22:59:46.381963 2212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:46 dist-test-slave-vxj2 krb5kdc[1549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316386, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:46.386346 2212 init.cc:377] Logged in from keytab as oryx/127.24.31.193@KRBTEST.COM (short username oryx)
I20260809 22:59:46.390652 2212 webserver.cc:460] Webserver started at http://127.24.31.193:35567/ using document root <none> and password file <none>
I20260809 22:59:46.391580 2212 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:46.391759 2212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:46.399313 2212 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 22:59:46.402843 2250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:46.404729 2212 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 22:59:46.404928 2212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "a92ff34637404f44bcd8afad4c60f29c"
format_stamp: "Formatted at 2026-08-09 22:59:43 on dist-test-slave-vxj2"
server_key: "1cb911482b409023149dabe010594e68"
server_key_iv: "c4fd0c988a41ff1d4ae04a5736c52534"
server_key_version: "encryptionkey@0"
I20260809 22:59:46.406389 2212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:46.428195 2212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:46.433620 2212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:46.433977 2212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:46.435245 2212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:46.437047 2212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:46.437206 2212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:46.437337 2212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:46.437426 2212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:46.476213 2212 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:34085
I20260809 22:59:46.476320 2377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:34085 every 8 connection(s)
I20260809 22:59:46.477825 2212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 22:59:46.480515 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 2212
I20260809 22:59:46.481302 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:37105
--local_ip_for_outbound_sockets=127.24.31.194
--tserver_master_addrs=127.24.31.254:45247
--webserver_port=38667
--webserver_interface=127.24.31.194
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.31.194
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:59:46 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:46 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:59:46.817644 2383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:46.817960 2383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:46.818125 2383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:46.825306 2383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:46.825440 2383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:46.825601 2383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 22:59:46.835184 2383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.194.keytab
--principal=oryx/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:37105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=38667
--tserver_master_addrs=127.24.31.254:45247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:46.836476 2383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:46.837850 2383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:46.856487 2392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:46.856552 2390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:46.856859 2389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:46.858057 2383 server_base.cc:1034] running on GCE node
I20260809 22:59:46.859169 2383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:46.860946 2383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:46.862255 2383 hybrid_clock.cc:648] HybridClock initialized: now 1786316386862125 us; error 91 us; skew 500 ppm
I20260809 22:59:46.862818 2383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:46 dist-test-slave-vxj2 krb5kdc[1549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316386, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:46.868650 2383 init.cc:377] Logged in from keytab as oryx/127.24.31.194@KRBTEST.COM (short username oryx)
I20260809 22:59:46.871062 2383 webserver.cc:460] Webserver started at http://127.24.31.194:38667/ using document root <none> and password file <none>
I20260809 22:59:46.872334 2383 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:46.872550 2383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:46.882575 2383 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 22:59:46.886900 2401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:46.889024 2383 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 22:59:46.889287 2383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "118c422e658d4e07909b0279bb4956e9"
format_stamp: "Formatted at 2026-08-09 22:59:44 on dist-test-slave-vxj2"
server_key: "7d1a9575f0d48420cc225832732aa35b"
server_key_iv: "b132ca9e194356836e1ace7e309e6f21"
server_key_version: "encryptionkey@0"
I20260809 22:59:46.891276 2383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:46.947134 2383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:46.954869 2383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:46.955310 2383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:46.957003 2383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:46.959441 2383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:46.959621 2383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:46.959772 2383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:46.959882 2383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:47.004673 2383 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:37105
I20260809 22:59:47.004668 2528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:37105 every 8 connection(s)
I20260809 22:59:47.007357 2383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 22:59:47.009572 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 2383
I20260809 22:59:47.010177 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:39565
--local_ip_for_outbound_sockets=127.24.31.195
--tserver_master_addrs=127.24.31.254:45247
--webserver_port=40561
--webserver_interface=127.24.31.195
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.31.195
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 22:59:47 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:47 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:59:47.357283 2533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 22:59:47.357592 2533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 22:59:47.357770 2533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 22:59:47.365036 2533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 22:59:47.365191 2533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 22:59:47.365379 2533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 22:59:47.375850 2533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/krb5kdc/oryx_127.24.31.195.keytab
--principal=oryx/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:39565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=40561
--tserver_master_addrs=127.24.31.254:45247
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 22:59:47.377260 2533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 22:59:47.378535 2533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 22:59:47.402784 2540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 22:59:47.413064 2541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:47.413561 2533 server_base.cc:1034] running on GCE node
W20260809 22:59:47.413874 2543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 22:59:47.414599 2533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 22:59:47.415675 2533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 22:59:47.416898 2533 hybrid_clock.cc:648] HybridClock initialized: now 1786316387416840 us; error 57 us; skew 500 ppm
I20260809 22:59:47.417377 2533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 22:59:47 dist-test-slave-vxj2 krb5kdc[1549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316387, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 22:59:47.420681 2533 init.cc:377] Logged in from keytab as oryx/127.24.31.195@KRBTEST.COM (short username oryx)
I20260809 22:59:47.422452 2533 webserver.cc:460] Webserver started at http://127.24.31.195:40561/ using document root <none> and password file <none>
I20260809 22:59:47.423173 2533 fs_manager.cc:359] Metadata directory not provided
I20260809 22:59:47.423338 2533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 22:59:47.428764 2533 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 22:59:47.431636 2550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:47.433058 2533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 22:59:47.433296 2533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "cf096cea0f244fb49eeaa7d08bd7e46d"
format_stamp: "Formatted at 2026-08-09 22:59:44 on dist-test-slave-vxj2"
server_key: "5b6cd44fb780a6aa8e83017a5bdf6e0d"
server_key_iv: "0177b428566618679c650f5d9e0b0f52"
server_key_version: "encryptionkey@0"
I20260809 22:59:47.434263 2533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 22:59:47.452186 2533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 22:59:47.456382 2533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 22:59:47.456670 2533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 22:59:47.457800 2533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 22:59:47.459249 2533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 22:59:47.459371 2533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:47.459501 2533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 22:59:47.459604 2533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 22:59:47.533942 2533 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:39565
I20260809 22:59:47.534005 2679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:39565 every 8 connection(s)
I20260809 22:59:47.535559 2533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 22:59:47.542548 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 2533
Aug 09 22:59:47 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:47 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:47.575064 2687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:47.546930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.24.31.254:45247)
0809 22:59:47.548073 (+ 1143us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:47.548082 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:47.548102 (+ 20us) server_negotiation.cc:407] Connection header received
0809 22:59:47.548152 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:47.548157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:47.548225 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:47.548339 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:47.550949 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.552667 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.559216 (+ 6549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.559905 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.560043 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:47.564099 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 22:59:47.564113 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 22:59:47.564136 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 22:59:47.564185 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 22:59:47.568790 (+ 4605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:47.569805 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:47.569818 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:47.569831 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:47.570064 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 22:59:47.570577 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 22:59:47.570586 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 22:59:47.570592 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 22:59:47.571304 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 22:59:47.571669 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0809 22:59:47.573375 (+ 1706us) server_negotiation.cc:299] Negotiation successful
0809 22:59:47.573866 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1015,"thread_start_us":887,"threads_started":1}
W20260809 22:59:48.195796 2381 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.700s user 0.002s sys 0.013s
W20260809 22:59:48.198287 2380 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:46.481007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:56549 (local address 127.24.31.254:45247)
0809 22:59:46.482148 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:46.482160 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:46.485904 (+ 3744us) server_negotiation.cc:407] Connection header received
0809 22:59:46.488062 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:46.488075 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:46.488382 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:46.488732 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:46.490262 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:46.491571 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:46.493109 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:46.494256 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:46.494442 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.197654 (+1703212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:56549: BlockingRecv error: recv got EOF from 127.24.31.193:56549 (error 108)
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":701,"threads_started":1}
I20260809 22:59:48.200593 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:46.481201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:56549)
0809 22:59:46.485712 (+ 4511us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:46.485758 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:46.487509 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:46.488868 (+ 1359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:46.488881 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:46.489534 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:46.490051 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:46.490068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:46.491729 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:46.491733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:46.492955 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:46.492968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:46.494482 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:46.494488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:46.494674 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:46.495367 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:46.495388 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:48.198706 (+1703318us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4263,"thread_start_us":4071,"threads_started":1}
W20260809 22:59:48.202286 2378 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:45247 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.31.254:45247: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:48.226338 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:48.203290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:33493)
0809 22:59:48.203779 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:48.203822 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:48.204010 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:48.204538 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:48.204546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:48.205254 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:48.205601 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.205617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.207942 (+ 2325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.207951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.210319 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.210343 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.212449 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.212459 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.212744 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.218207 (+ 5463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:48.218247 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:48.225869 (+ 7622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292}
W20260809 22:59:48.226689 2380 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:48.203435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:33493 (local address 127.24.31.254:45247)
0809 22:59:48.203731 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:48.203742 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:48.203929 (+ 187us) server_negotiation.cc:407] Connection header received
0809 22:59:48.204193 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:48.204202 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:48.204323 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:48.204479 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:48.205820 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.207710 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.210539 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.212170 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.212385 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.226234 (+ 13849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:33493: BlockingRecv error: recv got EOF from 127.24.31.193:33493 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 22:59:48.253602 2380 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:48.228428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:43137 (local address 127.24.31.254:45247)
0809 22:59:48.228689 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:48.228699 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:48.233664 (+ 4965us) server_negotiation.cc:407] Connection header received
0809 22:59:48.234042 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:48.234052 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:48.234195 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:48.234300 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:48.236032 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.238211 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.243002 (+ 4791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.243975 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.244111 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.253185 (+ 9074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:43137: BlockingRecv error: recv got EOF from 127.24.31.193:43137 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 22:59:48.254864 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:48.228143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:43137)
0809 22:59:48.233528 (+ 5385us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:48.233559 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:48.233828 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:48.234639 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:48.234648 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:48.235401 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:48.235771 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.235790 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.238528 (+ 2738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.238537 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.242278 (+ 3741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.242303 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.244522 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.244532 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.244865 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.246013 (+ 1148us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:48.246050 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:48.254399 (+ 8349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":5211,"thread_start_us":14944,"threads_started":1}
W20260809 22:59:48.255805 2378 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:59:48.473122 2532 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.432s user 0.003s sys 0.015s
I20260809 22:59:48.474835 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:47.014274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:38549)
0809 22:59:47.024172 (+ 9898us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:47.024230 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:47.027084 (+ 2854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:47.030477 (+ 3393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:47.030499 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:47.031657 (+ 1158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:47.032864 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:47.032902 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.034806 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.034814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:47.037958 (+ 3144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:47.037982 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.039328 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.039339 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:47.039809 (+ 470us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:47.040887 (+ 1078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:47.040934 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:48.473953 (+1433019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":9520,"thread_start_us":9239,"threads_started":1}
W20260809 22:59:48.474902 2531 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:47.013256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:38549 (local address 127.24.31.254:45247)
0809 22:59:47.029650 (+ 16394us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:47.029658 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:47.029685 (+ 27us) server_negotiation.cc:407] Connection header received
0809 22:59:47.029759 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:47.029767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:47.029876 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:47.030002 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:47.033173 (+ 3171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.034598 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.038181 (+ 3583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.039005 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.039170 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.474394 (+1435224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:38549: BlockingRecv error: recv got EOF from 127.24.31.194:38549 (error 108)
Metrics: {"server-negotiator.queue_time_us":16215,"thread_start_us":16093,"threads_started":1}
W20260809 22:59:48.476301 2529 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:45247 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.31.254:45247: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:48.489647 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:48.477200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:43175)
0809 22:59:48.477478 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:48.477507 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:48.477671 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:48.478083 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:48.478091 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:48.478805 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:48.479129 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.479145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.480291 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.480300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.482796 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.482817 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.484240 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.484245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.484376 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.485110 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:48.485126 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:48.489404 (+ 4278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 22:59:48.490249 2531 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:48.477581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:43175 (local address 127.24.31.254:45247)
0809 22:59:48.477753 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:48.477759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:48.477778 (+ 19us) server_negotiation.cc:407] Connection header received
0809 22:59:48.477834 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:48.477839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:48.477910 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:48.478010 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:48.479363 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.480125 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.483047 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.484050 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.484225 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.489910 (+ 5685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:43175: BlockingRecv error: recv got EOF from 127.24.31.194:43175 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:48.501794 2532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:48.491123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:49679)
0809 22:59:48.491284 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:48.491317 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:48.491399 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:48.491929 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:48.491933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:48.492247 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:48.492398 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.492405 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.493839 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.493848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.495275 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.495292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.496677 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.496683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.496854 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.497622 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:48.497640 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:48.501600 (+ 3960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 22:59:48.502166 2529 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 22:59:48.502830 2531 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:48.491471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:49679 (local address 127.24.31.254:45247)
0809 22:59:48.491640 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:48.491644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:48.491657 (+ 13us) server_negotiation.cc:407] Connection header received
0809 22:59:48.491699 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:48.491702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:48.491764 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:48.491895 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:48.492540 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.493605 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.495456 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.496492 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.496679 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.502424 (+ 5745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:49679: BlockingRecv error: recv got EOF from 127.24.31.194:49679 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 22:59:48.754761 2682 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.193s user 0.001s sys 0.017s
W20260809 22:59:48.755877 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:47.541070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:52369 (local address 127.24.31.254:45247)
0809 22:59:47.546955 (+ 5885us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:47.546962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:47.546983 (+ 21us) server_negotiation.cc:407] Connection header received
0809 22:59:47.547057 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:47.547066 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:47.547160 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:47.547314 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:47.552918 (+ 5604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.554583 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.558651 (+ 4068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.559344 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.559474 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.755680 (+1196206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:52369: BlockingRecv error: recv got EOF from 127.24.31.195:52369 (error 108)
Metrics: {"server-negotiator.queue_time_us":5770,"thread_start_us":5666,"threads_started":1}
I20260809 22:59:48.756613 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:47.540200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:52369)
0809 22:59:47.541242 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:47.541304 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:47.543876 (+ 2572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:47.548318 (+ 4442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:47.548332 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:47.549348 (+ 1016us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:47.550239 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:47.550269 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.554856 (+ 4587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.554867 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:47.558412 (+ 3545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:47.558433 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:47.559856 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:47.559867 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:47.560239 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:47.561511 (+ 1272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:47.561549 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:48.755472 (+1193923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":863,"threads_started":1}
W20260809 22:59:48.757581 2680 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:45247 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.31.254:45247: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:48.766942 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:48.758213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:55379)
0809 22:59:48.758414 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:48.758432 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:48.758523 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:48.758816 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:48.758819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:48.759145 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:48.759304 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.759311 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.760297 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.760301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.761388 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.761401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.762473 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.762480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.762653 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.763326 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:48.763344 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:48.766743 (+ 3399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 22:59:48.767349 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:48.758320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:55379 (local address 127.24.31.254:45247)
0809 22:59:48.758509 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:48.758513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:48.758531 (+ 18us) server_negotiation.cc:407] Connection header received
0809 22:59:48.758625 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:48.758628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:48.758689 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:48.758769 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:48.759428 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.760173 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.761560 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.762264 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.762408 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.767064 (+ 4656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:55379: BlockingRecv error: recv got EOF from 127.24.31.195:55379 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:48 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:48.775640 2682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:48.767724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:45819)
0809 22:59:48.767856 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:48.767876 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:48.767961 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:48.768333 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:48.768336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:48.768607 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:48.768743 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.768750 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.769766 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.769769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.770686 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:48.770696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.771376 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.771380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:48.771502 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.771972 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:48.771988 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:48.775461 (+ 3473us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 22:59:48.775890 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:48.767817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:45819 (local address 127.24.31.254:45247)
0809 22:59:48.768057 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:48.768061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:48.768078 (+ 17us) server_negotiation.cc:407] Connection header received
0809 22:59:48.768121 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:48.768125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:48.768198 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:48.768298 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:48.768889 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.769641 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.770820 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:48.771239 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:48.771338 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:48.775634 (+ 4296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:45819: BlockingRecv error: recv got EOF from 127.24.31.195:45819 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
W20260809 22:59:48.776007 2680 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 22:59:49 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:49 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:49.265609 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:49.256363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:46135)
0809 22:59:49.256882 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:49.256898 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:49.257064 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:49.257452 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:49.257457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:49.257934 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:49.258085 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:49.258094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.259085 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.259091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:49.260090 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:49.260102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.260849 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.260856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:49.261088 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:49.261790 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:49.261808 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:49.265226 (+ 3418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":277,"threads_started":1}
W20260809 22:59:49.265647 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:49.256463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:46135 (local address 127.24.31.254:45247)
0809 22:59:49.256704 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:49.256712 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:49.256994 (+ 282us) server_negotiation.cc:407] Connection header received
0809 22:59:49.257221 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:49.257227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:49.257310 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:49.257417 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:49.258237 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.258940 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.260249 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.260667 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.260820 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:49.265442 (+ 4622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:46135: BlockingRecv error: recv got EOF from 127.24.31.193:46135 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 22:59:49 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:49 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:49.512400 2691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:49.502743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:44307)
0809 22:59:49.503298 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:49.503314 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:49.503493 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:49.503840 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:49.503844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:49.504295 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:49.504439 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:49.504457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.505664 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.505669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:49.506644 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:49.506655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.507391 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.507397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:49.507580 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:49.508308 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:49.508337 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:49.512029 (+ 3692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":322,"threads_started":1}
W20260809 22:59:49.512432 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:49.502840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:44307 (local address 127.24.31.254:45247)
0809 22:59:49.503105 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:49.503113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:49.503408 (+ 295us) server_negotiation.cc:407] Connection header received
0809 22:59:49.503623 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:49.503631 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:49.503699 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:49.503807 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:49.504600 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.505496 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.506789 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.507209 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.507349 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:49.512239 (+ 4890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:44307: BlockingRecv error: recv got EOF from 127.24.31.194:44307 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 09 22:59:49 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:49 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:49.785874 2692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:49.776576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:42411)
0809 22:59:49.777181 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:49.777202 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:49.777353 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:49.777658 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:49.777661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:49.778091 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:49.778251 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:49.778259 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.779209 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.779212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:49.780160 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:49.780171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.781077 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.781082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:49.781230 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:49.781868 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:49.781888 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:49.785636 (+ 3748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":343,"threads_started":1}
W20260809 22:59:49.786103 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:49.776724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:42411 (local address 127.24.31.254:45247)
0809 22:59:49.776896 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:49.776901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:49.777303 (+ 402us) server_negotiation.cc:407] Connection header received
0809 22:59:49.777451 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:49.777454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:49.777517 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:49.777609 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:49.778379 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.779089 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.780310 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:49.780867 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:49.780973 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:49.785852 (+ 4879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:42411: BlockingRecv error: recv got EOF from 127.24.31.195:42411 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 22:59:50 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:50 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:50.276036 2693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:50.266784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:34093)
0809 22:59:50.267321 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:50.267337 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:50.267518 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:50.267875 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:50.267880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:50.268313 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:50.268498 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:50.268508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.269642 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.269647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:50.270587 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:50.270600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.271318 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.271324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:50.271513 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:50.272113 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:50.272131 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:50.275696 (+ 3565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":307,"threads_started":1}
W20260809 22:59:50.276170 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:50.266971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:34093 (local address 127.24.31.254:45247)
0809 22:59:50.267159 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:50.267166 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:50.267430 (+ 264us) server_negotiation.cc:407] Connection header received
0809 22:59:50.267648 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:50.267654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:50.267732 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:50.267833 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:50.268647 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.269495 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.270738 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.271158 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.271293 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:50.275894 (+ 4601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:34093: BlockingRecv error: recv got EOF from 127.24.31.193:34093 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 22:59:50 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:50 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:50.523491 2694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:50.513530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:55787)
0809 22:59:50.514057 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:50.514073 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:50.514236 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:50.514589 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:50.514593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:50.515043 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:50.515215 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:50.515223 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.516361 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.516367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:50.517312 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:50.517324 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.518049 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.518055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:50.518233 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:50.518809 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:50.518826 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:50.523088 (+ 4262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":291,"threads_started":1}
W20260809 22:59:50.523552 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:50.513604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:55787 (local address 127.24.31.254:45247)
0809 22:59:50.513887 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:50.513895 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:50.514162 (+ 267us) server_negotiation.cc:407] Connection header received
0809 22:59:50.514374 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:50.514380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:50.514451 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:50.514556 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:50.515366 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.516202 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.517465 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.517886 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.518017 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:50.523289 (+ 5272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:55787: BlockingRecv error: recv got EOF from 127.24.31.194:55787 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 22:59:50 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:50 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:50.795728 2695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:50.786802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:32875)
0809 22:59:50.787333 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:50.787354 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:50.787529 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:50.787903 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:50.787910 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:50.788325 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:50.788513 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:50.788522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.789575 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.789584 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:50.790498 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:50.790510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.791201 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.791208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:50.791389 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:50.791994 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:50.792011 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:50.795483 (+ 3472us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":299,"threads_started":1}
W20260809 22:59:50.796053 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:50.786984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:32875 (local address 127.24.31.254:45247)
0809 22:59:50.787207 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:50.787214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:50.787437 (+ 223us) server_negotiation.cc:407] Connection header received
0809 22:59:50.787653 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:50.787665 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:50.787750 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:50.787853 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:50.788654 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.789451 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.790661 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:50.791054 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:50.791185 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:50.795718 (+ 4533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:32875: BlockingRecv error: recv got EOF from 127.24.31.195:32875 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 22:59:51 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:51 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:51.286751 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:51.277479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:34661)
0809 22:59:51.278012 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:51.278028 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:51.278141 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:51.278472 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:51.278475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:51.278864 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:51.279043 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:51.279051 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.280123 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.280127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:51.281047 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:51.281059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.281814 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.281819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:51.281956 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:51.282533 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:51.282550 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:51.286357 (+ 3807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":294,"threads_started":1}
W20260809 22:59:51.286891 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:51.277523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:34661 (local address 127.24.31.254:45247)
0809 22:59:51.277706 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:51.277711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:51.278115 (+ 404us) server_negotiation.cc:407] Connection header received
0809 22:59:51.278261 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:51.278265 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:51.278331 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:51.278421 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:51.279177 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.279940 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.281244 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.281683 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.281785 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:51.286637 (+ 4852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:34661: BlockingRecv error: recv got EOF from 127.24.31.193:34661 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 22:59:51 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:51 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:51.534320 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:51.524685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:44439)
0809 22:59:51.525125 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:51.525142 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:51.525430 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:51.525805 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:51.525809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:51.526191 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:51.526366 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:51.526374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.527470 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.527475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:51.529014 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:51.529030 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.529863 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.529868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:51.529988 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:51.530542 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:51.530561 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:51.534078 (+ 3517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":247,"threads_started":1}
W20260809 22:59:51.534598 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:51.524744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:44439 (local address 127.24.31.254:45247)
0809 22:59:51.525018 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:51.525025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:51.525380 (+ 355us) server_negotiation.cc:407] Connection header received
0809 22:59:51.525538 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:51.525541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:51.525601 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:51.525718 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:51.526528 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.527304 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.529219 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.529727 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.529824 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:51.534332 (+ 4508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:44439: BlockingRecv error: recv got EOF from 127.24.31.194:44439 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 09 22:59:51 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:51 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:51.806461 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:51.796652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:60033)
0809 22:59:51.797271 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:51.797291 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:51.797444 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:51.797819 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:51.797824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:51.798215 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:51.798445 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:51.798460 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.799500 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.799508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:51.800437 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:51.800451 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.801212 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.801218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:51.801394 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:51.802042 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:51.802059 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:51.806169 (+ 4110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":326,"threads_started":1}
W20260809 22:59:51.806615 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:51.796751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:60033 (local address 127.24.31.254:45247)
0809 22:59:51.796989 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:51.796998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:51.797371 (+ 373us) server_negotiation.cc:407] Connection header received
0809 22:59:51.797571 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:51.797583 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:51.797659 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:51.797767 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:51.798607 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.799356 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.800594 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:51.801023 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:51.801159 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:51.806383 (+ 5224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:60033: BlockingRecv error: recv got EOF from 127.24.31.195:60033 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 22:59:52 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:52 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:52.297336 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:52.287982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:55433)
0809 22:59:52.288479 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:52.288496 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:52.288647 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:52.289020 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:52.289024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:52.289475 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:52.289639 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:52.289648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.290673 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.290679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:52.291712 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:52.291723 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.292444 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.292450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:52.292628 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:52.293412 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:52.293429 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:52.297023 (+ 3594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":265,"threads_started":1}
W20260809 22:59:52.297557 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:52.288051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:55433 (local address 127.24.31.254:45247)
0809 22:59:52.288311 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:52.288319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:52.288582 (+ 263us) server_negotiation.cc:407] Connection header received
0809 22:59:52.288772 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:52.288777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:52.288860 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:52.288967 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:52.289788 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.290526 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.291864 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.292299 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.292430 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:52.297294 (+ 4864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:55433: BlockingRecv error: recv got EOF from 127.24.31.193:55433 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 22:59:52 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:52 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:52.545308 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:52.535559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:60921)
0809 22:59:52.536103 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:52.536120 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:52.536277 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:52.536642 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:52.536647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:52.537055 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:52.537291 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:52.537302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.538355 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.538359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:52.539393 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:52.539407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.540500 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.540506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:52.540665 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:52.541435 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:52.541459 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:52.544976 (+ 3517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":294,"threads_started":1}
W20260809 22:59:52.545591 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:52.535668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:60921 (local address 127.24.31.254:45247)
0809 22:59:52.535896 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:52.535904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:52.536207 (+ 303us) server_negotiation.cc:407] Connection header received
0809 22:59:52.536405 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:52.536410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:52.536486 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:52.536590 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:52.537450 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.538199 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.539612 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.540319 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.540436 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:52.545295 (+ 4859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:60921: BlockingRecv error: recv got EOF from 127.24.31.194:60921 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 22:59:52 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:52 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:52.816677 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:52.807610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:51039)
0809 22:59:52.808194 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:52.808215 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:52.808346 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:52.808668 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:52.808671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:52.809075 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:52.809270 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:52.809281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.810248 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.810251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:52.811254 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:52.811265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.811932 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.811939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:52.812155 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:52.812816 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:52.812833 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:52.816293 (+ 3460us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":351,"threads_started":1}
W20260809 22:59:52.816759 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:52.807698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:51039 (local address 127.24.31.254:45247)
0809 22:59:52.807865 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:52.807870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:52.808287 (+ 417us) server_negotiation.cc:407] Connection header received
0809 22:59:52.808444 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:52.808449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:52.808532 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:52.808616 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:52.809394 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.810122 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.811374 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:52.811764 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:52.811915 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:52.816483 (+ 4568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:51039: BlockingRecv error: recv got EOF from 127.24.31.195:51039 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 22:59:53 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:53 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:53.312826 2703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:53.298354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:50851)
0809 22:59:53.298904 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:53.298925 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:53.299055 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:53.299368 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:53.299372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:53.299903 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:53.300074 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:53.300083 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.301197 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.301202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:53.302179 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:53.302191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.302959 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.302963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:53.303105 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:53.308515 (+ 5410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:53.308533 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:53.312495 (+ 3962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":303,"threads_started":1}
W20260809 22:59:53.312968 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:53.298538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:50851 (local address 127.24.31.254:45247)
0809 22:59:53.298718 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:53.298725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:53.299023 (+ 298us) server_negotiation.cc:407] Connection header received
0809 22:59:53.299160 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:53.299164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:53.299242 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:53.299337 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:53.300211 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.300991 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.302348 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.302807 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.302924 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:53.312762 (+ 9838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:50851: BlockingRecv error: recv got EOF from 127.24.31.193:50851 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 22:59:53 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:53 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:53.555721 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:53.546395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:40345)
0809 22:59:53.546976 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:53.546992 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:53.547150 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:53.547500 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:53.547505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:53.547974 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:53.548119 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:53.548127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.549315 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.549320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:53.550281 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:53.550293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.551170 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.551176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:53.551360 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:53.551941 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:53.551958 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:53.555378 (+ 3420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":331,"threads_started":1}
W20260809 22:59:53.555833 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:53.546582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:40345 (local address 127.24.31.254:45247)
0809 22:59:53.546776 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:53.546782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:53.547079 (+ 297us) server_negotiation.cc:407] Connection header received
0809 22:59:53.547273 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:53.547281 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:53.547360 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:53.547464 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:53.548283 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.549124 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.550509 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.551013 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.551159 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:53.555587 (+ 4428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:40345: BlockingRecv error: recv got EOF from 127.24.31.194:40345 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 22:59:53 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:53 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:53.826413 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:53.817644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:48177)
0809 22:59:53.818192 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:53.818212 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:53.818325 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:53.818620 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:53.818624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:53.819067 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:53.819210 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:53.819218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.820175 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.820179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:53.821081 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:53.821091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.821804 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.821808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:53.821927 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:53.822566 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:53.822583 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:53.826192 (+ 3609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":315,"threads_started":1}
W20260809 22:59:53.826617 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:53.817763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:48177 (local address 127.24.31.254:45247)
0809 22:59:53.817958 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:53.817963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:53.818283 (+ 320us) server_negotiation.cc:407] Connection header received
0809 22:59:53.818419 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:53.818422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:53.818498 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:53.818582 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:53.819342 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.820060 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.821277 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:53.821684 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:53.821784 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:53.826375 (+ 4591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:48177: BlockingRecv error: recv got EOF from 127.24.31.195:48177 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 22:59:54 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:54 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:54.323267 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:54.313752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:44057)
0809 22:59:54.314254 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:54.314274 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:54.314393 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:54.314694 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:54.314698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:54.315149 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:54.315310 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:54.315319 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.316876 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.316883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:54.317984 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:54.317996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.318749 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.318753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:54.318890 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:54.319470 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:54.319487 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:54.323039 (+ 3552us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":273,"threads_started":1}
W20260809 22:59:54.323493 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:54.313876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:44057 (local address 127.24.31.254:45247)
0809 22:59:54.314150 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:54.314156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:54.314356 (+ 200us) server_negotiation.cc:407] Connection header received
0809 22:59:54.314487 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:54.314491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:54.314551 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:54.314629 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:54.315460 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.316668 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.318133 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.318553 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.318684 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:54.323247 (+ 4563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:44057: BlockingRecv error: recv got EOF from 127.24.31.193:44057 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 09 22:59:54 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:54 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:54.566268 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:54.557044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:42701)
0809 22:59:54.557695 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:54.557723 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:54.557876 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:54.558308 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:54.558312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:54.558885 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:54.559042 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:54.559051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.560080 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.560084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:54.561073 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:54.561084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.561834 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.561838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:54.561962 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:54.562535 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:54.562553 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:54.565984 (+ 3431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":397,"threads_started":1}
W20260809 22:59:54.566449 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:54.557098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:42701 (local address 127.24.31.254:45247)
0809 22:59:54.557400 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:54.557410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:54.557821 (+ 411us) server_negotiation.cc:407] Connection header received
0809 22:59:54.558019 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:54.558037 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:54.558127 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:54.558251 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:54.559176 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.559948 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.561255 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.561695 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.561798 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:54.566249 (+ 4451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:42701: BlockingRecv error: recv got EOF from 127.24.31.194:42701 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
Aug 09 22:59:54 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:54 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:54.837405 2709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:54.827449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:57141)
0809 22:59:54.827957 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:54.827979 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:54.828145 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:54.828494 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:54.828500 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:54.828983 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:54.829199 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:54.829217 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.830294 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.830299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:54.831249 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:54.831260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.832001 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.832007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:54.832191 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:54.832828 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:54.832884 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:54.837009 (+ 4125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":276,"threads_started":1}
W20260809 22:59:54.837539 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:54.827514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:57141 (local address 127.24.31.254:45247)
0809 22:59:54.827759 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:54.827768 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:54.828069 (+ 301us) server_negotiation.cc:407] Connection header received
0809 22:59:54.828270 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:54.828276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:54.828356 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:54.828459 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:54.829372 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.830131 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.831413 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:54.831828 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:54.831964 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:54.837254 (+ 5290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:57141: BlockingRecv error: recv got EOF from 127.24.31.195:57141 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 22:59:55 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:55 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:55.334197 2711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:55.324449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:40683)
0809 22:59:55.325332 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:55.325356 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:55.325507 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:55.325899 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:55.325907 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:55.326510 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:55.326726 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:55.326735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.327728 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.327732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:55.328755 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:55.328767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.329556 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.329561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:55.329683 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:55.330341 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:55.330358 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:55.333774 (+ 3416us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":475,"threads_started":1}
W20260809 22:59:55.334369 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:55.324580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:40683 (local address 127.24.31.254:45247)
0809 22:59:55.324956 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:55.324966 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:55.325449 (+ 483us) server_negotiation.cc:407] Connection header received
0809 22:59:55.325647 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:55.325652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:55.325733 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:55.325830 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:55.326858 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.327601 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.328881 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.329398 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.329506 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:55.334136 (+ 4630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:40683: BlockingRecv error: recv got EOF from 127.24.31.193:40683 (error 108)
Metrics: {"server-negotiator.queue_time_us":196}
Aug 09 22:59:55 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:55 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:55.576709 2712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:55.567250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:42195)
0809 22:59:55.567925 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:55.567958 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:55.568114 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:55.568522 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:55.568528 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:55.569119 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:55.569406 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:55.569420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.570554 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.570558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:55.571545 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:55.571557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.572225 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.572229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:55.572342 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:55.572919 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:55.572936 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:55.576452 (+ 3516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":318,"threads_started":1}
W20260809 22:59:55.576937 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:55.567387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:42195 (local address 127.24.31.254:45247)
0809 22:59:55.567644 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:55.567650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:55.568054 (+ 404us) server_negotiation.cc:407] Connection header received
0809 22:59:55.568246 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:55.568251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:55.568335 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:55.568444 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:55.569574 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.570423 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.571679 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.572092 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.572186 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:55.576661 (+ 4475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:42195: BlockingRecv error: recv got EOF from 127.24.31.194:42195 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 09 22:59:55 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:55 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:55.847388 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:55.838429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:56279)
0809 22:59:55.838982 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:55.839002 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:55.839115 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:55.839411 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:55.839414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:55.839823 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:55.840004 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:55.840013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.840978 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.840982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:55.841979 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:55.841991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.842640 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.842644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:55.842752 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:55.843296 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:55.843313 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:55.847129 (+ 3816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":240,"threads_started":1}
W20260809 22:59:55.847512 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:55.838453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:56279 (local address 127.24.31.254:45247)
0809 22:59:55.838681 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:55.838691 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:55.839087 (+ 396us) server_negotiation.cc:407] Connection header received
0809 22:59:55.839213 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:55.839217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:55.839292 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:55.839371 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:55.840128 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.840857 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.842108 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:55.842514 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:55.842608 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:55.847309 (+ 4701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:56279: BlockingRecv error: recv got EOF from 127.24.31.195:56279 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 22:59:56 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:56 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:56.344789 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:56.335115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:40975)
0809 22:59:56.335625 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:56.335641 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:56.335754 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:56.336173 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:56.336177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:56.336666 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:56.336912 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:56.336934 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.338395 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.338400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:56.339400 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:56.339412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.340079 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.340084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:56.340197 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:56.340749 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:56.340772 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:56.344424 (+ 3652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":254,"threads_started":1}
W20260809 22:59:56.344847 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:56.335273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:40975 (local address 127.24.31.254:45247)
0809 22:59:56.335491 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:56.335497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:56.335715 (+ 218us) server_negotiation.cc:407] Connection header received
0809 22:59:56.335889 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:56.335896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:56.335985 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:56.336109 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:56.337309 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.338259 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.339526 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.339916 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.340018 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:56.344618 (+ 4600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:40975: BlockingRecv error: recv got EOF from 127.24.31.193:40975 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 22:59:56 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:56 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:56.586969 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:56.577794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:39105)
0809 22:59:56.578338 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:56.578355 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:56.578466 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:56.578776 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:56.578779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:56.579198 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:56.579379 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:56.579388 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.580551 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.580554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:56.581507 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:56.581518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.582201 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.582206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:56.582319 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:56.582960 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:56.582978 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:56.586727 (+ 3749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":275,"threads_started":1}
W20260809 22:59:56.587169 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:56.577883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:39105 (local address 127.24.31.254:45247)
0809 22:59:56.578071 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:56.578075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:56.578428 (+ 353us) server_negotiation.cc:407] Connection header received
0809 22:59:56.578561 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:56.578565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:56.578654 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:56.578734 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:56.579509 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.580425 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.581627 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.582043 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.582136 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:56.586943 (+ 4807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:39105: BlockingRecv error: recv got EOF from 127.24.31.194:39105 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 22:59:56 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:56 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:56.857834 2716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:56.848345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:46307)
0809 22:59:56.848908 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:56.848930 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:56.849054 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:56.849399 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:56.849402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:56.849890 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:56.850070 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:56.850079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.851146 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.851149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:56.852084 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:56.852095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.852807 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.852811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:56.852916 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:56.853517 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:56.853533 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:56.857465 (+ 3932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":320,"threads_started":1}
W20260809 22:59:56.857949 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:56.848413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:46307 (local address 127.24.31.254:45247)
0809 22:59:56.848673 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:56.848681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:56.849023 (+ 342us) server_negotiation.cc:407] Connection header received
0809 22:59:56.849196 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:56.849207 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:56.849276 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:56.849357 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:56.850273 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.851024 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.852234 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:56.852655 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:56.852758 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:56.857679 (+ 4921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:46307: BlockingRecv error: recv got EOF from 127.24.31.195:46307 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 22:59:57 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:57 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:57.355227 2717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:57.345976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:34349)
0809 22:59:57.346539 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:57.346556 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:57.346674 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:57.346968 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:57.346972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:57.347453 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:57.347604 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:57.347611 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.348723 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.348727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:57.349812 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:57.349823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.350477 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.350481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:57.350599 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:57.351157 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:57.351174 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:57.354989 (+ 3815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":303,"threads_started":1}
W20260809 22:59:57.355406 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:57.346048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:34349 (local address 127.24.31.254:45247)
0809 22:59:57.346330 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:57.346336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:57.346621 (+ 285us) server_negotiation.cc:407] Connection header received
0809 22:59:57.346756 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:57.346761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:57.346832 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:57.346912 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:57.347733 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.348597 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.349931 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.350337 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.350434 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:57.355141 (+ 4707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:34349: BlockingRecv error: recv got EOF from 127.24.31.193:34349 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 22:59:57 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:57 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:57.597018 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:57.587927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:45075)
0809 22:59:57.588463 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:57.588481 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:57.588594 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:57.588931 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:57.588935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:57.589393 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:57.589550 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:57.589559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.590553 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.590556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:57.591532 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:57.591544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.592239 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.592244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:57.592383 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:57.593002 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:57.593027 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:57.596789 (+ 3762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":257,"threads_started":1}
W20260809 22:59:57.597263 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:57.587987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:45075 (local address 127.24.31.254:45247)
0809 22:59:57.588225 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:57.588231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:57.588563 (+ 332us) server_negotiation.cc:407] Connection header received
0809 22:59:57.588680 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:57.588683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:57.588746 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:57.588827 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:57.589678 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.590412 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.591654 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.592080 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.592177 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:57.596980 (+ 4803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:45075: BlockingRecv error: recv got EOF from 127.24.31.194:45075 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 22:59:57 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:57 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:57.867761 2720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:57.858865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:40399)
0809 22:59:57.859388 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:57.859409 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:57.859583 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:57.859937 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:57.859941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:57.860372 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:57.860520 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:57.860529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.861552 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.861556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:57.862547 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:57.862559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.863252 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.863257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:57.863395 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:57.863975 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:57.863993 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:57.867522 (+ 3529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":290,"threads_started":1}
W20260809 22:59:57.867933 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:57.858902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:40399 (local address 127.24.31.254:45247)
0809 22:59:57.859144 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:57.859153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:57.859502 (+ 349us) server_negotiation.cc:407] Connection header received
0809 22:59:57.859693 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:57.859696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:57.859763 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:57.859843 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:57.860701 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.861421 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.862670 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:57.863087 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:57.863180 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:57.867718 (+ 4538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:40399: BlockingRecv error: recv got EOF from 127.24.31.195:40399 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 22:59:58 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:58 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:58.368827 2721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:58.356212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:35285)
0809 22:59:58.356726 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:58.356742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:58.356898 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:58.357295 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:58.357300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:58.357711 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:58.357864 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:58.357873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.359039 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.359044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:58.360045 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:58.360057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.360770 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.360776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:58.360964 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:58.364870 (+ 3906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:58.364888 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:58.368481 (+ 3593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":265,"threads_started":1}
W20260809 22:59:58.368952 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:58.356312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:35285 (local address 127.24.31.254:45247)
0809 22:59:58.356558 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:58.356565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:58.356832 (+ 267us) server_negotiation.cc:407] Connection header received
0809 22:59:58.357045 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:58.357054 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:58.357115 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:58.357269 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:58.358021 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.358879 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.360196 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.360619 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.360753 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:58.368666 (+ 7913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:35285: BlockingRecv error: recv got EOF from 127.24.31.193:35285 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 22:59:58 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:58 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:58.607291 2722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:58.598062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:37015)
0809 22:59:58.598656 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:58.598673 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:58.598830 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:58.599214 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:58.599217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:58.599621 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:58.599854 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:58.599866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.601016 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.601022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:58.602056 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:58.602068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.602756 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.602762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:58.602937 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:58.603526 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:58.603543 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:58.607019 (+ 3476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":360,"threads_started":1}
W20260809 22:59:58.607420 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:58.598221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:37015 (local address 127.24.31.254:45247)
0809 22:59:58.598435 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:58.598443 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:58.598761 (+ 318us) server_negotiation.cc:407] Connection header received
0809 22:59:58.598976 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:58.598983 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:58.599059 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:58.599180 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:58.600022 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.600859 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.602202 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.602602 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.602738 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:58.607215 (+ 4477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:37015: BlockingRecv error: recv got EOF from 127.24.31.194:37015 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 22:59:58 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:58 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:58.877748 2723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:58.868732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:37803)
0809 22:59:58.869310 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:58.869331 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:58.869450 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:58.869748 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:58.869751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:58.870115 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:58.870277 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:58.870284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.871361 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.871364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:58.872314 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:58.872325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.873027 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.873031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:58.873141 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:58.873732 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:58.873748 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:58.877410 (+ 3662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":333,"threads_started":1}
W20260809 22:59:58.877880 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:58.868977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:37803 (local address 127.24.31.254:45247)
0809 22:59:58.869233 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:58.869240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:58.869419 (+ 179us) server_negotiation.cc:407] Connection header received
0809 22:59:58.869544 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:58.869548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:58.869624 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:58.869712 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:58.870421 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.871203 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.872451 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:58.872897 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:58.873004 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:58.877609 (+ 4605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:37803: BlockingRecv error: recv got EOF from 127.24.31.195:37803 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 09 22:59:59 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:59 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:59.379379 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:59.370020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:54083)
0809 22:59:59.370607 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:59.370624 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:59.370786 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:59.371240 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:59.371245 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:59.371698 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:59.371936 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:59.371946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.372919 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.372924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:59.373952 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:59.373965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.374654 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.374659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:59.374827 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:59.375477 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:59.375495 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:59.379148 (+ 3653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":357,"threads_started":1}
W20260809 22:59:59.379554 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:59.370107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:54083 (local address 127.24.31.254:45247)
0809 22:59:59.370352 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:59.370361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:59.370711 (+ 350us) server_negotiation.cc:407] Connection header received
0809 22:59:59.370921 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:59.370929 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:59.371021 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:59.371125 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:59.372069 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.372772 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.374089 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.374523 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.374614 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:59.379324 (+ 4710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:54083: BlockingRecv error: recv got EOF from 127.24.31.193:54083 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 09 22:59:59 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:59 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:59.618192 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:59.608359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:55065)
0809 22:59:59.608861 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:59.608878 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:59.608998 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:59.609321 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:59.609324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:59.609935 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:59.610207 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:59.610219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.611694 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.611699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:59.612797 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:59.612810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.613505 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.613510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:59.613648 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:59.614414 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:59.614434 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:59.617946 (+ 3512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":248,"threads_started":1}
W20260809 22:59:59.618353 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:59.608420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:55065 (local address 127.24.31.254:45247)
0809 22:59:59.608653 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:59.608660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:59.608966 (+ 306us) server_negotiation.cc:407] Connection header received
0809 22:59:59.609101 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:59.609104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:59.609180 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:59.609259 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:59.610367 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.611468 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.612928 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.613339 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.613433 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:59.618133 (+ 4700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:55065: BlockingRecv error: recv got EOF from 127.24.31.194:55065 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 22:59:59 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 22:59:59 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 22:59:59.888705 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 22:59:59.878821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:46477)
0809 22:59:59.879383 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 22:59:59.879409 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 22:59:59.879535 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 22:59:59.879874 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 22:59:59.879882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 22:59:59.880381 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 22:59:59.880596 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:59.880606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.881631 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.881635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:59.882657 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 22:59:59.882669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.883338 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.883343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 22:59:59.883477 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:59.884070 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 22:59:59.884089 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 22:59:59.888373 (+ 4284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":319,"threads_started":1}
W20260809 22:59:59.888873 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 22:59:59.878876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:46477 (local address 127.24.31.254:45247)
0809 22:59:59.879046 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 22:59:59.879050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 22:59:59.879492 (+ 442us) server_negotiation.cc:407] Connection header received
0809 22:59:59.879639 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 22:59:59.879643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 22:59:59.879705 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 22:59:59.879785 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 22:59:59.880729 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.881472 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.882795 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 22:59:59.883187 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 22:59:59.883277 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 22:59:59.888637 (+ 5360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:46477: BlockingRecv error: recv got EOF from 127.24.31.195:46477 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 23:00:00 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:00:00 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:00:00.390436 2727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:00.380532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:40545)
0809 23:00:00.381092 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:00.381110 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:00.381386 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:00.381711 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:00.381715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:00.382183 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:00.382392 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:00.382402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.383411 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.383415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:00.384421 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:00.384433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.385218 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.385225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:00.385442 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:00.386100 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:00.386119 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:00.390139 (+ 4020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":341,"threads_started":1}
W20260809 23:00:00.390537 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:00:00.380567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:40545 (local address 127.24.31.254:45247)
0809 23:00:00.380821 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:00.380830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:00.381292 (+ 462us) server_negotiation.cc:407] Connection header received
0809 23:00:00.381504 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:00.381508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:00.381572 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:00.381655 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:00.382530 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.383273 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.384551 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.384985 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.385074 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:00.390321 (+ 5247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:40545: BlockingRecv error: recv got EOF from 127.24.31.193:40545 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 23:00:00 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:00:00 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:00:00.628638 2728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:00.619215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:54857)
0809 23:00:00.619787 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:00.619806 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:00.619936 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:00.620257 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:00.620260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:00.620704 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:00.620904 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:00.620915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.622037 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.622042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:00.623185 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:00.623202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.623960 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.623965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:00.624148 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:00.624716 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:00.624733 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:00.628374 (+ 3641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":335,"threads_started":1}
W20260809 23:00:00.628842 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:00:00.619402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:54857 (local address 127.24.31.254:45247)
0809 23:00:00.619650 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:00.619656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:00.619894 (+ 238us) server_negotiation.cc:407] Connection header received
0809 23:00:00.620074 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:00.620077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:00.620141 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:00.620224 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:00.621055 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.621898 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.623342 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.623784 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.623876 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:00.628565 (+ 4689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:54857: BlockingRecv error: recv got EOF from 127.24.31.194:54857 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 09 23:00:00 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:00:00 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:00:00.899747 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:00.889801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:51067)
0809 23:00:00.890333 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:00.890354 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:00.890480 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:00.890881 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:00.890889 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:00.891361 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:00.891542 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:00.891551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.892640 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.892644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:00.893627 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:00.893639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.894562 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.894567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:00.894730 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:00.895403 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:00.895458 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:00.899429 (+ 3971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":282,"threads_started":1}
W20260809 23:00:00.899991 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:00:00.889878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:51067 (local address 127.24.31.254:45247)
0809 23:00:00.890129 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:00.890135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:00.890436 (+ 301us) server_negotiation.cc:407] Connection header received
0809 23:00:00.890623 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:00.890627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:00.890693 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:00.890842 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:00.891681 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.892515 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.893747 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:00.894361 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:00.894459 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:00.899729 (+ 5270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:51067: BlockingRecv error: recv got EOF from 127.24.31.195:51067 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 23:00:01 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:00:01 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:00:01.401383 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:01.391984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:45509)
0809 23:00:01.392535 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:01.392551 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:01.392668 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:01.392987 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:01.392990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:01.393470 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:01.393622 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:01.393630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.394844 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.394850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:01.395829 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:01.395841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.396621 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.396626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:01.396761 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:01.397440 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:01.397465 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:01.401090 (+ 3625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":303,"threads_started":1}
W20260809 23:00:01.401641 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:00:01.392100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:45509 (local address 127.24.31.254:45247)
0809 23:00:01.392330 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:01.392335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:01.392631 (+ 296us) server_negotiation.cc:407] Connection header received
0809 23:00:01.392767 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:01.392771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:01.392837 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:01.392926 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:01.393753 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.394701 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.395991 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.396477 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.396586 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:01.401367 (+ 4781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:45509: BlockingRecv error: recv got EOF from 127.24.31.193:45509 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 23:00:01 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:00:01 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.194@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 23:00:01.644711 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:00:01.630055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:41263 (local address 127.24.31.254:45247)
0809 23:00:01.630251 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:01.630256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:01.630496 (+ 240us) server_negotiation.cc:407] Connection header received
0809 23:00:01.630625 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:01.630628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:01.630689 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:01.630773 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:01.631597 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.632344 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.633636 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.634064 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.634155 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:01.644504 (+ 10349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.194:41263: BlockingRecv error: recv got EOF from 127.24.31.194:41263 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 23:00:01.645236 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:01.629857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.194:41263)
0809 23:00:01.630394 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:01.630418 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:01.630538 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:01.630897 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:01.630901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:01.631311 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:01.631463 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:01.631471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.632503 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.632508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:01.633492 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:01.633505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.634251 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.634256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:01.634393 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:01.639705 (+ 5312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:01.639737 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:01.644958 (+ 5221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":305,"threads_started":1}
Aug 09 23:00:01 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:00:01 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.195@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:00:01.910992 2733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:01.900806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.195:38897)
0809 23:00:01.901402 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:01.901426 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:01.901595 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:01.902081 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:01.902089 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:01.902705 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:01.902890 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:01.902899 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.903992 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.903996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:01.904895 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:01.904905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.905902 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.905909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:01.906132 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:01.906924 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:01.906947 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:01.910696 (+ 3749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":359,"threads_started":1}
W20260809 23:00:01.911231 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:00:01.900966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:38897 (local address 127.24.31.254:45247)
0809 23:00:01.901255 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:01.901262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:01.901518 (+ 256us) server_negotiation.cc:407] Connection header received
0809 23:00:01.901826 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:01.901831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:01.901902 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:01.902002 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:01.903061 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.903848 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.905074 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:01.905711 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:01.905868 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:01.911006 (+ 5138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.195:38897: BlockingRecv error: recv got EOF from 127.24.31.195:38897 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 23:00:02 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:00:02 dist-test-slave-vxj2 krb5kdc[1549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.31.193@KRBTEST.COM for oryx/127.24.31.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:00:02.412387 2734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:02.402514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:45247 (local address 127.24.31.193:38993)
0809 23:00:02.403059 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:02.403076 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:02.403192 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:02.403529 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:02.403532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:02.403917 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:02.404122 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:02.404130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.405276 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.405280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:02.406207 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:02.406219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.407018 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.407025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:02.407233 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:02.408015 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:02.408033 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:02.412024 (+ 3991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.31.254:45247: Server oryx/127.24.31.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":276,"threads_started":1}
W20260809 23:00:02.412566 2684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:00:02.402609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:38993 (local address 127.24.31.254:45247)
0809 23:00:02.402889 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:02.402894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:02.403157 (+ 263us) server_negotiation.cc:407] Connection header received
0809 23:00:02.403293 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:02.403297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:02.403371 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:02.403459 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:02.404301 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.405058 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.406350 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.406839 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.406990 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:02.412253 (+ 5263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.31.193:38993: BlockingRecv error: recv got EOF from 127.24.31.193:38993 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
I20260809 23:00:02.553404 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2212
I20260809 23:00:02.567610 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:02.554986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36490 (local address 127.24.31.193:34085)
0809 23:00:02.555624 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:02.555632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:02.555666 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:00:02.555746 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:02.555753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:02.556010 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:02.556198 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:02.557829 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.559031 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.560381 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.561021 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.561139 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:02.563866 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:02.563876 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:02.563892 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:02.563946 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:02.566913 (+ 2967us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.31.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 23:00:02.567297 (+ 384us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36490: Request ticket server kudu/127.24.31.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":280,"threads_started":1}
W20260809 23:00:02.567706 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2212: Not authorized: Client connection negotiation failed: client connection to 127.24.31.193:34085: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.31.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 23:00:02.567737 2736 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36490: Request ticket server kudu/127.24.31.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 23:00:02.567986 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 2212
I20260809 23:00:02.587803 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2383
I20260809 23:00:02.602026 2737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:02.589199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35322 (local address 127.24.31.194:37105)
0809 23:00:02.589893 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:02.589903 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:02.590210 (+ 307us) server_negotiation.cc:407] Connection header received
0809 23:00:02.590352 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:02.590365 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:02.590681 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:02.590939 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:02.592232 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.593476 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.594857 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.595492 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.595611 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:02.598218 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:02.598227 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:02.598242 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:02.598293 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:02.601418 (+ 3125us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.31.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 23:00:02.601745 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35322: Request ticket server kudu/127.24.31.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":293,"threads_started":1}
W20260809 23:00:02.602166 2737 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35322: Request ticket server kudu/127.24.31.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 23:00:02.602190 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2383: Not authorized: Client connection negotiation failed: client connection to 127.24.31.194:37105: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.31.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 23:00:02.602326 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 2383
I20260809 23:00:02.622246 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2533
I20260809 23:00:02.636399 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:02.623485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.24.31.195:39565)
0809 23:00:02.624147 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:02.624155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:02.624185 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:00:02.624318 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:02.624328 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:02.624688 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:02.624976 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:02.627261 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.628352 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.629637 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:02.630264 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:02.630386 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:02.632991 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:02.632999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:02.633014 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:02.633057 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:02.635823 (+ 2766us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.31.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 23:00:02.636125 (+ 302us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41642: Request ticket server kudu/127.24.31.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":344,"threads_started":1}
W20260809 23:00:02.636561 2738 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41642: Request ticket server kudu/127.24.31.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 23:00:02.636528 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2533: Not authorized: Client connection negotiation failed: client connection to 127.24.31.195:39565: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.31.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 23:00:02.636725 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 2533
I20260809 23:00:02.659060 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2132
I20260809 23:00:02.660506 2205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:02.782691 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 2132
2026-08-09T23:00:02Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21471 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T23:00:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:00:02Z Disabled control of system clock
I20260809 23:00:02.868988 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:35487
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:35487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 23:00:03.206720 2744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:03.207032 2744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:03.207207 2744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:03.214829 2744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:00:03.214952 2744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:03.215049 2744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:03.215154 2744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:00:03.215236 2744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:00:03.224829 2744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:35487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:35487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:03.226307 2744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:03.228192 2744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:03.240876 2750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:03.240862 2749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:03.240859 2752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:03.241547 2744 server_base.cc:1034] running on GCE node
I20260809 23:00:03.242388 2744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:03.243870 2744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:03.245107 2744 hybrid_clock.cc:648] HybridClock initialized: now 1786316403245061 us; error 43 us; skew 500 ppm
I20260809 23:00:03.245512 2744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:00:03.248095 2744 webserver.cc:460] Webserver started at http://127.24.31.254:34783/ using document root <none> and password file <none>
I20260809 23:00:03.248831 2744 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:03.248971 2744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:03.249367 2744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:03.252398 2744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "77666e216df248069439f9be5ec25ab7"
format_stamp: "Formatted at 2026-08-09 23:00:03 on dist-test-slave-vxj2"
server_key: "4491ea0ce5389405a619e168e7fe9519"
server_key_iv: "5d7e84f7b5dce9c944703319e259c42b"
server_key_version: "encryptionkey@0"
I20260809 23:00:03.253281 2744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "77666e216df248069439f9be5ec25ab7"
format_stamp: "Formatted at 2026-08-09 23:00:03 on dist-test-slave-vxj2"
server_key: "4491ea0ce5389405a619e168e7fe9519"
server_key_iv: "5d7e84f7b5dce9c944703319e259c42b"
server_key_version: "encryptionkey@0"
I20260809 23:00:03.258504 2744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:00:03.262104 2759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:03.263633 2744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:00:03.263837 2744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "77666e216df248069439f9be5ec25ab7"
format_stamp: "Formatted at 2026-08-09 23:00:03 on dist-test-slave-vxj2"
server_key: "4491ea0ce5389405a619e168e7fe9519"
server_key_iv: "5d7e84f7b5dce9c944703319e259c42b"
server_key_version: "encryptionkey@0"
I20260809 23:00:03.264089 2744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:03.276439 2744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:03.277390 2744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:03.277717 2744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:03.296536 2744 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:35487
I20260809 23:00:03.296545 2817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:35487 every 8 connection(s)
I20260809 23:00:03.297966 2744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:00:03.301661 2818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:00:03.304384 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 2744
I20260809 23:00:03.304592 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:00:03.305438 24703 external_mini_cluster.cc:1428] Setting key 6ebbc026cf12be2f8c33cb42cdd4bf33
I20260809 23:00:03.315240 2818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7: Bootstrap starting.
I20260809 23:00:03.319046 2818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7: Neither blocks nor log segments found. Creating new log.
I20260809 23:00:03.320252 2818 log.cc:824] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7: Log is configured to *not* fsync() on all Append() calls
I20260809 23:00:03.320806 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:03.307877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60290 (local address 127.24.31.254:35487)
0809 23:00:03.309145 (+ 1268us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:03.309207 (+ 62us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:03.309271 (+ 64us) server_negotiation.cc:407] Connection header received
0809 23:00:03.311205 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:03.311234 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:03.311749 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:03.312113 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:03.312775 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:03.314183 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:03.316068 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:03.316769 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:03.316918 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:03.318280 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:03.318288 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:03.318300 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:03.318835 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:03.318917 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:03.318939 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:03.319038 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:03.319323 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 23:00:03.319694 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":411,"threads_started":1}
I20260809 23:00:03.323449 2818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7: No bootstrap required, opened a new log
I20260809 23:00:03.330833 2818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77666e216df248069439f9be5ec25ab7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35487 } }
I20260809 23:00:03.331157 2818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:00:03.331286 2818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77666e216df248069439f9be5ec25ab7, State: Initialized, Role: FOLLOWER
I20260809 23:00:03.331830 2818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77666e216df248069439f9be5ec25ab7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35487 } }
I20260809 23:00:03.332034 2818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:00:03.332134 2818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:00:03.332252 2818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:00:03.334525 2818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77666e216df248069439f9be5ec25ab7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35487 } }
I20260809 23:00:03.335103 2818 leader_election.cc:302] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77666e216df248069439f9be5ec25ab7; no voters:
I20260809 23:00:03.335734 2818 leader_election.cc:288] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:00:03.335878 2823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:00:03.336658 2823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [term 1 LEADER]: Becoming Leader. State: Replica: 77666e216df248069439f9be5ec25ab7, State: Running, Role: LEADER
I20260809 23:00:03.337263 2823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77666e216df248069439f9be5ec25ab7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35487 } }
I20260809 23:00:03.338003 2818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:00:03.340426 2824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77666e216df248069439f9be5ec25ab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77666e216df248069439f9be5ec25ab7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35487 } } }
I20260809 23:00:03.340417 2825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77666e216df248069439f9be5ec25ab7. Latest consensus state: current_term: 1 leader_uuid: "77666e216df248069439f9be5ec25ab7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77666e216df248069439f9be5ec25ab7" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35487 } } }
I20260809 23:00:03.340875 2824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [sys.catalog]: This master's current role is: LEADER
I20260809 23:00:03.340941 2825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7 [sys.catalog]: This master's current role is: LEADER
I20260809 23:00:03.343286 2831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:00:03.348910 2831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:00:03.358989 2831 catalog_manager.cc:1347] Generated new cluster ID: 682da2a62201460d84665e67a711406b
I20260809 23:00:03.359149 2831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:00:03.401274 2831 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:00:03.402597 2831 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:00:03.409390 2831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77666e216df248069439f9be5ec25ab7: Generated new TSK 0
I20260809 23:00:03.410261 2831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:00:03.424909 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:35487
--builtin_ntp_servers=127.24.31.212:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 23:00:03.765970 2843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:03.766251 2843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:03.766340 2843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:03.773772 2843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:03.773875 2843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:03.773980 2843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:00:03.783178 2843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:35487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:03.784382 2843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:03.786098 2843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:03.799149 2852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:03.799113 2848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:03.799188 2849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:03.800073 2843 server_base.cc:1034] running on GCE node
I20260809 23:00:03.800724 2843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:03.801749 2843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:03.802989 2843 hybrid_clock.cc:648] HybridClock initialized: now 1786316403802940 us; error 44 us; skew 500 ppm
I20260809 23:00:03.803316 2843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:00:03.810591 2843 webserver.cc:460] Webserver started at http://127.24.31.193:42635/ using document root <none> and password file <none>
I20260809 23:00:03.811368 2843 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:03.811507 2843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:03.811846 2843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:03.814774 2843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "28e08bb525ba471e857a87a000cf0e4a"
format_stamp: "Formatted at 2026-08-09 23:00:03 on dist-test-slave-vxj2"
server_key: "642e4ec7ee1206584b31fbd023632378"
server_key_iv: "84c21366934983fb8ac0d322ffc8830b"
server_key_version: "encryptionkey@0"
I20260809 23:00:03.815593 2843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "28e08bb525ba471e857a87a000cf0e4a"
format_stamp: "Formatted at 2026-08-09 23:00:03 on dist-test-slave-vxj2"
server_key: "642e4ec7ee1206584b31fbd023632378"
server_key_iv: "84c21366934983fb8ac0d322ffc8830b"
server_key_version: "encryptionkey@0"
I20260809 23:00:03.820767 2843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 23:00:03.824190 2859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:03.825544 2843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:00:03.825721 2843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "28e08bb525ba471e857a87a000cf0e4a"
format_stamp: "Formatted at 2026-08-09 23:00:03 on dist-test-slave-vxj2"
server_key: "642e4ec7ee1206584b31fbd023632378"
server_key_iv: "84c21366934983fb8ac0d322ffc8830b"
server_key_version: "encryptionkey@0"
I20260809 23:00:03.825935 2843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:03.841966 2843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:03.842919 2843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:03.843222 2843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:03.844169 2843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:00:03.845674 2843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:00:03.845796 2843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:03.845932 2843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:00:03.846019 2843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:03.880328 2843 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:40915
I20260809 23:00:03.880373 2988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:40915 every 8 connection(s)
I20260809 23:00:03.881696 2843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:00:03.891839 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 2843
I20260809 23:00:03.892053 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:00:03.892717 24703 external_mini_cluster.cc:1428] Setting key 4e0464edc4382c72611bd1fa09490952
I20260809 23:00:03.894622 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:03.884266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:44723 (local address 127.24.31.254:35487)
0809 23:00:03.884724 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:03.884731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:03.885942 (+ 1211us) server_negotiation.cc:407] Connection header received
0809 23:00:03.887734 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:03.887739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:03.887850 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:03.887951 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:03.889036 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:03.889872 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:03.891475 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:03.891917 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:03.892122 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:03.893672 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:03.893677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:03.893680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:03.893698 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:03.893762 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:03.893774 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:03.893889 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:03.894115 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 23:00:03.894210 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":205,"threads_started":1}
I20260809 23:00:03.895702 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:03.884827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:35487 (local address 127.24.31.193:44723)
0809 23:00:03.885787 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:03.885829 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:03.887456 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:03.888174 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:03.888182 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:03.888267 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:03.888848 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:03.888867 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:03.890017 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:03.890020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:03.891263 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:03.891299 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:03.892308 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:03.892316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:03.892523 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:03.893329 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:00:03.893332 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:03.893371 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:00:03.893451 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:00:03.893899 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:00:03.893909 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:00:03.893916 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:00:03.894071 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 23:00:03.894377 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":346,"threads_started":1}
I20260809 23:00:03.896063 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:35487
--builtin_ntp_servers=127.24.31.212:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 23:00:03.897540 2989 heartbeater.cc:347] Connected to a master server at 127.24.31.254:35487
I20260809 23:00:03.897886 2989 heartbeater.cc:464] Registering TS with master...
I20260809 23:00:03.898842 2989 heartbeater.cc:511] Master 127.24.31.254:35487 requested a full tablet report, sending...
I20260809 23:00:03.901254 2778 ts_manager.cc:194] Registered new tserver with Master: 28e08bb525ba471e857a87a000cf0e4a (127.24.31.193:40915)
I20260809 23:00:03.903424 2778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.193:44723
W20260809 23:00:04.231724 2994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:04.232064 2994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:04.232205 2994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:04.239799 2994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:04.239913 2994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:04.240022 2994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:00:04.249352 2994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:35487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:04.250717 2994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:04.252521 2994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:04.265717 3000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:04.265919 3001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:04.266204 3004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:04.266672 2994 server_base.cc:1034] running on GCE node
I20260809 23:00:04.267324 2994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:04.268416 2994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:04.269649 2994 hybrid_clock.cc:648] HybridClock initialized: now 1786316404269585 us; error 58 us; skew 500 ppm
I20260809 23:00:04.270004 2994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:00:04.272224 2994 webserver.cc:460] Webserver started at http://127.24.31.194:36885/ using document root <none> and password file <none>
I20260809 23:00:04.272938 2994 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:04.273072 2994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:04.273443 2994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:04.276264 2994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "43a17b45aef34fc38d8808ec70cab8e3"
format_stamp: "Formatted at 2026-08-09 23:00:04 on dist-test-slave-vxj2"
server_key: "db40a42b7dcda9a1cec2769649587433"
server_key_iv: "88684b9a6839b69ac03308a41623a3a7"
server_key_version: "encryptionkey@0"
I20260809 23:00:04.277072 2994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "43a17b45aef34fc38d8808ec70cab8e3"
format_stamp: "Formatted at 2026-08-09 23:00:04 on dist-test-slave-vxj2"
server_key: "db40a42b7dcda9a1cec2769649587433"
server_key_iv: "88684b9a6839b69ac03308a41623a3a7"
server_key_version: "encryptionkey@0"
I20260809 23:00:04.282383 2994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 23:00:04.285804 3011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:04.287118 2994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:00:04.287295 2994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "43a17b45aef34fc38d8808ec70cab8e3"
format_stamp: "Formatted at 2026-08-09 23:00:04 on dist-test-slave-vxj2"
server_key: "db40a42b7dcda9a1cec2769649587433"
server_key_iv: "88684b9a6839b69ac03308a41623a3a7"
server_key_version: "encryptionkey@0"
I20260809 23:00:04.287525 2994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:04.305706 2994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:04.306679 2994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:04.306985 2994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:04.308014 2994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:00:04.309803 2994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:00:04.309918 2994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:04.310016 2994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:00:04.310084 2994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:04.344753 2994 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:41379
I20260809 23:00:04.344805 3140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:41379 every 8 connection(s)
I20260809 23:00:04.346117 2994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:00:04.354983 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 2994
I20260809 23:00:04.355168 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:00:04.355754 24703 external_mini_cluster.cc:1428] Setting key f16a8e0157e7838be4e85cbc63725e19
I20260809 23:00:04.358829 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:35487
--builtin_ntp_servers=127.24.31.212:42299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 23:00:04.361889 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:04.348827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:58693 (local address 127.24.31.254:35487)
0809 23:00:04.349020 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:04.349027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:04.350859 (+ 1832us) server_negotiation.cc:407] Connection header received
0809 23:00:04.353712 (+ 2853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:04.353719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:04.353840 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:04.353987 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:04.355457 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.356487 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.358784 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.359492 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.359642 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:04.360939 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:04.360947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:04.360955 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:04.360985 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:04.361027 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:04.361045 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:04.361140 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:04.361404 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 23:00:04.361511 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 23:00:04.362733 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:04.349488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:35487 (local address 127.24.31.194:58693)
0809 23:00:04.350650 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:04.350709 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:04.353377 (+ 2668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:04.354277 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:04.354291 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:04.354406 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:04.355221 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:04.355251 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.356639 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.356644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:04.358627 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:04.358641 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.359700 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.359705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:04.359892 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:04.360621 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:00:04.360624 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:04.360662 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:00:04.360732 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:00:04.361145 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:00:04.361192 (+ 47us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:00:04.361199 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:00:04.361354 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 23:00:04.361629 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":334,"threads_started":1}
I20260809 23:00:04.364425 3141 heartbeater.cc:347] Connected to a master server at 127.24.31.254:35487
I20260809 23:00:04.364761 3141 heartbeater.cc:464] Registering TS with master...
I20260809 23:00:04.365615 3141 heartbeater.cc:511] Master 127.24.31.254:35487 requested a full tablet report, sending...
I20260809 23:00:04.367069 2778 ts_manager.cc:194] Registered new tserver with Master: 43a17b45aef34fc38d8808ec70cab8e3 (127.24.31.194:41379)
I20260809 23:00:04.368053 2778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.194:58693
W20260809 23:00:04.695924 3145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:04.696200 3145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:04.696347 3145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:04.703617 3145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:04.703717 3145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:04.703912 3145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:00:04.712939 3145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:35487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:04.714286 3145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:04.715998 3145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:04.729216 3152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:04.729146 3155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:04.729439 3151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:04.729928 3145 server_base.cc:1034] running on GCE node
I20260809 23:00:04.730562 3145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:04.731444 3145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:04.732651 3145 hybrid_clock.cc:648] HybridClock initialized: now 1786316404732583 us; error 58 us; skew 500 ppm
I20260809 23:00:04.732951 3145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:00:04.735159 3145 webserver.cc:460] Webserver started at http://127.24.31.195:33379/ using document root <none> and password file <none>
I20260809 23:00:04.735807 3145 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:04.735893 3145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:04.736140 3145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:04.739123 3145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "e54bb4a6db4a4c9b9bcca00e3ed46aaa"
format_stamp: "Formatted at 2026-08-09 23:00:04 on dist-test-slave-vxj2"
server_key: "8b85a1bf299d94749ca365dd6441dd67"
server_key_iv: "ce3ba84c315bf48e8a2fbdd26a5a926f"
server_key_version: "encryptionkey@0"
I20260809 23:00:04.739861 3145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "e54bb4a6db4a4c9b9bcca00e3ed46aaa"
format_stamp: "Formatted at 2026-08-09 23:00:04 on dist-test-slave-vxj2"
server_key: "8b85a1bf299d94749ca365dd6441dd67"
server_key_iv: "ce3ba84c315bf48e8a2fbdd26a5a926f"
server_key_version: "encryptionkey@0"
I20260809 23:00:04.744889 3145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 23:00:04.748241 3162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:04.749614 3145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:00:04.749751 3145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "e54bb4a6db4a4c9b9bcca00e3ed46aaa"
format_stamp: "Formatted at 2026-08-09 23:00:04 on dist-test-slave-vxj2"
server_key: "8b85a1bf299d94749ca365dd6441dd67"
server_key_iv: "ce3ba84c315bf48e8a2fbdd26a5a926f"
server_key_version: "encryptionkey@0"
I20260809 23:00:04.749923 3145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:04.771962 3145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:04.772782 3145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:04.773067 3145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:04.774117 3145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:00:04.775569 3145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:00:04.775655 3145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:04.775727 3145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:00:04.775780 3145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:04.809886 3145 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:45033
I20260809 23:00:04.809998 3291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:45033 every 8 connection(s)
I20260809 23:00:04.811178 3145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:00:04.816154 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 3145
I20260809 23:00:04.816298 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:00:04.816784 24703 external_mini_cluster.cc:1428] Setting key a1af8b9503b7be5eb6894ff74e6bf74d
I20260809 23:00:04.824657 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:04.813861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:55981 (local address 127.24.31.254:35487)
0809 23:00:04.814070 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:04.814074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:04.815426 (+ 1352us) server_negotiation.cc:407] Connection header received
0809 23:00:04.817525 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:04.817550 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:04.817653 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:04.817774 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:04.819189 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.820131 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.821815 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.822213 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.822317 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:04.823861 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:04.823867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:04.823877 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:04.823921 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:04.823956 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:04.823969 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:04.824057 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:04.824302 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 23:00:04.824390 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 23:00:04.825754 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:04.814351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:35487 (local address 127.24.31.195:55981)
0809 23:00:04.815241 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:04.815302 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:04.817191 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:04.818075 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:04.818085 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:04.818224 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:04.819011 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:04.819040 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.820309 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.820313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:04.821665 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:04.821678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.822463 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.822469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:04.822690 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:04.823516 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:00:04.823518 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:04.823559 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:00:04.823648 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:00:04.824065 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:00:04.824074 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:00:04.824079 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:00:04.824240 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 23:00:04.824563 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":272,"threads_started":1}
I20260809 23:00:04.827343 3292 heartbeater.cc:347] Connected to a master server at 127.24.31.254:35487
I20260809 23:00:04.827692 3292 heartbeater.cc:464] Registering TS with master...
I20260809 23:00:04.828516 3292 heartbeater.cc:511] Master 127.24.31.254:35487 requested a full tablet report, sending...
I20260809 23:00:04.830050 2778 ts_manager.cc:194] Registered new tserver with Master: e54bb4a6db4a4c9b9bcca00e3ed46aaa (127.24.31.195:45033)
I20260809 23:00:04.831058 2778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.195:55981
I20260809 23:00:04.832667 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 23:00:04.839193 2991 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:00:04.837265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50186 (local address 127.24.31.254:35487)
0809 23:00:04.837505 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:04.837514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:04.837980 (+ 466us) server_negotiation.cc:407] Connection header received
0809 23:00:04.838198 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:04.838202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:04.838274 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:04.838381 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:04.838911 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50186: BlockingRecv error: recv got EOF from 127.0.0.1:50186 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 23:00:04.840219 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2843
I20260809 23:00:04.848096 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:04.841095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53966 (local address 127.24.31.193:40915)
0809 23:00:04.841806 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:04.841813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:04.841837 (+ 24us) server_negotiation.cc:407] Connection header received
0809 23:00:04.841905 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:04.841910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:04.842116 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:04.842243 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:04.842651 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.843706 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.845064 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.845646 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.845742 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:04.846690 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:04.846697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:04.846709 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:04.847275 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:04.847324 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:04.847343 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:04.847437 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:04.847565 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 23:00:04.847688 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":459,"threads_started":1}
I20260809 23:00:04.849457 2983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:04.930214 2989 heartbeater.cc:503] Master 127.24.31.254:35487 was elected leader, sending a full tablet report...
I20260809 23:00:04.932789 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 2843
I20260809 23:00:04.952477 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2994
I20260809 23:00:04.960357 3304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:04.953440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.24.31.194:41379)
0809 23:00:04.954009 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:04.954016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:04.954043 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:00:04.954146 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:04.954158 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:04.954463 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:04.954597 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:04.955024 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.956013 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.957281 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:04.957840 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:04.957939 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:04.958988 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:04.958996 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:04.959009 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:04.959605 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:04.959659 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:04.959679 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:04.959768 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:04.959936 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 23:00:04.960070 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":300,"threads_started":1}
I20260809 23:00:04.961645 3136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:05.043614 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 2994
I20260809 23:00:05.064134 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 3145
I20260809 23:00:05.072763 3306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:05.065287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46016 (local address 127.24.31.195:45033)
0809 23:00:05.065927 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:05.065934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:05.065979 (+ 45us) server_negotiation.cc:407] Connection header received
0809 23:00:05.066041 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:05.066047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:05.066265 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:05.066446 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:05.066980 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:05.068083 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:05.069429 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:05.070040 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:05.070183 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:05.071289 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:05.071297 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:05.071311 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:05.071895 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:05.071952 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:05.071972 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:05.072097 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:05.072271 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 23:00:05.072405 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":313,"threads_started":1}
I20260809 23:00:05.074090 3287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:05.157359 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 3145
I20260809 23:00:05.175985 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 2744
I20260809 23:00:05.177253 2812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:05.281881 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 2744
2026-08-09T23:00:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2493 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T23:00:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:00:05Z Disabled control of system clock
I20260809 23:00:05.363816 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44905
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:42219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:44905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:00:05.703730 3312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:05.704042 3312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:05.704133 3312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:05.711591 3312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:00:05.711719 3312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:05.711757 3312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:05.711815 3312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:00:05.711867 3312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:00:05.721319 3312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:44905
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:05.722602 3312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:05.724416 3312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:05.736061 3320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:05.736115 3319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:05.736136 3322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:05.738431 3312 server_base.cc:1034] running on GCE node
I20260809 23:00:05.739190 3312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:05.740707 3312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:05.741937 3312 hybrid_clock.cc:648] HybridClock initialized: now 1786316405741890 us; error 56 us; skew 500 ppm
I20260809 23:00:05.742285 3312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:00:05.744652 3312 webserver.cc:460] Webserver started at http://127.24.31.254:38275/ using document root <none> and password file <none>
I20260809 23:00:05.745433 3312 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:05.745566 3312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:05.745913 3312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:05.750921 3312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "17019635a4044bc0888b73648c40744b"
format_stamp: "Formatted at 2026-08-09 23:00:05 on dist-test-slave-vxj2"
server_key: "e4b05af7d6159a8157720d016e3408cf"
server_key_iv: "4099b475e556f64f2e3f64967cc6cd99"
server_key_version: "encryptionkey@0"
I20260809 23:00:05.751745 3312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "17019635a4044bc0888b73648c40744b"
format_stamp: "Formatted at 2026-08-09 23:00:05 on dist-test-slave-vxj2"
server_key: "e4b05af7d6159a8157720d016e3408cf"
server_key_iv: "4099b475e556f64f2e3f64967cc6cd99"
server_key_version: "encryptionkey@0"
I20260809 23:00:05.756911 3312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 23:00:05.760351 3329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:05.761643 3312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:00:05.761824 3312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "17019635a4044bc0888b73648c40744b"
format_stamp: "Formatted at 2026-08-09 23:00:05 on dist-test-slave-vxj2"
server_key: "e4b05af7d6159a8157720d016e3408cf"
server_key_iv: "4099b475e556f64f2e3f64967cc6cd99"
server_key_version: "encryptionkey@0"
I20260809 23:00:05.762053 3312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:05.767925 3312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:05.768705 3312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:05.769033 3312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:05.788086 3312 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:44905
I20260809 23:00:05.788120 3391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:44905 every 8 connection(s)
I20260809 23:00:05.789517 3312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:00:05.791049 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 3312
I20260809 23:00:05.791296 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:00:05.792114 24703 external_mini_cluster.cc:1428] Setting key ce9a70ddfc3fb0ab7d58272b441e22e5
I20260809 23:00:05.794553 3392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:00:05.800671 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:05.794349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.24.31.254:44905)
0809 23:00:05.795412 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:05.795423 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:05.795466 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:00:05.797052 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:05.797074 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:05.797566 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:05.797904 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:05.799038 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:05.799044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:05.799056 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:05.799074 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:05.799147 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:05.799163 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:05.799357 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:05.799473 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 23:00:05.799764 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":437,"threads_started":1}
I20260809 23:00:05.807017 3392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b: Bootstrap starting.
I20260809 23:00:05.810249 3392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b: Neither blocks nor log segments found. Creating new log.
I20260809 23:00:05.811231 3392 log.cc:824] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:00:05.814199 3392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b: No bootstrap required, opened a new log
I20260809 23:00:05.821589 3392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17019635a4044bc0888b73648c40744b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44905 } }
I20260809 23:00:05.821931 3392 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:00:05.822016 3392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17019635a4044bc0888b73648c40744b, State: Initialized, Role: FOLLOWER
I20260809 23:00:05.822494 3392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17019635a4044bc0888b73648c40744b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44905 } }
I20260809 23:00:05.822690 3392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:00:05.822780 3392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:00:05.822901 3392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:00:05.824949 3392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17019635a4044bc0888b73648c40744b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44905 } }
I20260809 23:00:05.825480 3392 leader_election.cc:302] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17019635a4044bc0888b73648c40744b; no voters:
I20260809 23:00:05.826099 3392 leader_election.cc:288] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:00:05.826272 3397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:00:05.827704 3397 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [term 1 LEADER]: Becoming Leader. State: Replica: 17019635a4044bc0888b73648c40744b, State: Running, Role: LEADER
I20260809 23:00:05.828341 3392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:00:05.828284 3397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17019635a4044bc0888b73648c40744b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44905 } }
I20260809 23:00:05.831364 3399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17019635a4044bc0888b73648c40744b. Latest consensus state: current_term: 1 leader_uuid: "17019635a4044bc0888b73648c40744b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17019635a4044bc0888b73648c40744b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44905 } } }
I20260809 23:00:05.831837 3398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17019635a4044bc0888b73648c40744b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17019635a4044bc0888b73648c40744b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44905 } } }
I20260809 23:00:05.832319 3399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [sys.catalog]: This master's current role is: LEADER
I20260809 23:00:05.832366 3398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b [sys.catalog]: This master's current role is: LEADER
I20260809 23:00:05.833765 3407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:00:05.838707 3407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:00:05.848201 3407 catalog_manager.cc:1347] Generated new cluster ID: d459b8e62efa48729b0c2e7485a18879
I20260809 23:00:05.848321 3407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:00:05.876833 3407 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:00:05.877319 3407 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:00:05.884971 3407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17019635a4044bc0888b73648c40744b: Generated new TSK 0
I20260809 23:00:05.885848 3407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:00:05.897380 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44905
--builtin_ntp_servers=127.24.31.212:42219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:00:06.241282 3416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:06.241566 3416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:06.241724 3416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:06.248941 3416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:06.249044 3416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:06.249248 3416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:00:06.258961 3416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:06.260430 3416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:06.262261 3416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:06.275424 3421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:06.275463 3424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:06.275661 3422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:06.276355 3416 server_base.cc:1034] running on GCE node
I20260809 23:00:06.276981 3416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:06.277951 3416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:06.279162 3416 hybrid_clock.cc:648] HybridClock initialized: now 1786316406279092 us; error 67 us; skew 500 ppm
I20260809 23:00:06.279480 3416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:00:06.281687 3416 webserver.cc:460] Webserver started at http://127.24.31.193:34667/ using document root <none> and password file <none>
I20260809 23:00:06.282356 3416 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:06.282444 3416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:06.282754 3416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:06.285590 3416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "d7c2b99323a748d9b195033195afff6e"
format_stamp: "Formatted at 2026-08-09 23:00:06 on dist-test-slave-vxj2"
server_key: "dc6e6f8185b1bd641c893cff2e22f769"
server_key_iv: "646ce8949168f8de7410f85ed6de04a0"
server_key_version: "encryptionkey@0"
I20260809 23:00:06.286401 3416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "d7c2b99323a748d9b195033195afff6e"
format_stamp: "Formatted at 2026-08-09 23:00:06 on dist-test-slave-vxj2"
server_key: "dc6e6f8185b1bd641c893cff2e22f769"
server_key_iv: "646ce8949168f8de7410f85ed6de04a0"
server_key_version: "encryptionkey@0"
I20260809 23:00:06.291510 3416 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:00:06.294936 3431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:06.296229 3416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:00:06.296399 3416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "d7c2b99323a748d9b195033195afff6e"
format_stamp: "Formatted at 2026-08-09 23:00:06 on dist-test-slave-vxj2"
server_key: "dc6e6f8185b1bd641c893cff2e22f769"
server_key_iv: "646ce8949168f8de7410f85ed6de04a0"
server_key_version: "encryptionkey@0"
I20260809 23:00:06.296655 3416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:06.302276 3416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:06.303040 3416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:06.303315 3416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:06.304291 3416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:00:06.305814 3416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:00:06.305902 3416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:06.306057 3416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:00:06.306155 3416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:06.340096 3416 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:42615
I20260809 23:00:06.340148 3564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:42615 every 8 connection(s)
I20260809 23:00:06.341513 3416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:00:06.347371 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 3416
I20260809 23:00:06.347620 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:00:06.348555 24703 external_mini_cluster.cc:1428] Setting key f64445abaf9b974e36a316d50408dd43
I20260809 23:00:06.349996 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:06.344192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:39963 (local address 127.24.31.254:44905)
0809 23:00:06.344655 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:06.344659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:06.345730 (+ 1071us) server_negotiation.cc:407] Connection header received
0809 23:00:06.347474 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:06.347480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:06.347555 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:06.347676 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:06.349082 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:06.349088 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:06.349093 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:06.349125 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:06.349221 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:06.349238 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:06.349388 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:06.349588 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 23:00:06.349661 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":263,"threads_started":1}
I20260809 23:00:06.350857 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:06.344636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44905 (local address 127.24.31.193:39963)
0809 23:00:06.345570 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:06.345607 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:06.347196 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:06.347920 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:06.347932 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:06.348026 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:06.348772 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:00:06.348774 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:06.348814 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:00:06.348897 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:00:06.349351 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:00:06.349365 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:00:06.349374 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 23:00:06.349666 (+ 292us) client_negotiation.cc:240] Negotiation successful
0809 23:00:06.349913 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":265,"threads_started":1}
I20260809 23:00:06.351699 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44905
--builtin_ntp_servers=127.24.31.212:42219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 23:00:06.352479 3565 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44905
I20260809 23:00:06.352785 3565 heartbeater.cc:464] Registering TS with master...
I20260809 23:00:06.353456 3565 heartbeater.cc:511] Master 127.24.31.254:44905 requested a full tablet report, sending...
I20260809 23:00:06.355468 3347 ts_manager.cc:194] Registered new tserver with Master: d7c2b99323a748d9b195033195afff6e (127.24.31.193:42615)
W20260809 23:00:06.682826 3569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:06.683115 3569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:06.683282 3569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:06.690688 3569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:06.690791 3569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:06.690963 3569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:00:06.700340 3569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:06.701843 3569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:06.703696 3569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:06.717476 3574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:06.717417 3575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:06.717363 3577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:06.718473 3569 server_base.cc:1034] running on GCE node
I20260809 23:00:06.719099 3569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:06.720048 3569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:06.721249 3569 hybrid_clock.cc:648] HybridClock initialized: now 1786316406721204 us; error 65 us; skew 500 ppm
I20260809 23:00:06.721580 3569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:00:06.723816 3569 webserver.cc:460] Webserver started at http://127.24.31.194:37773/ using document root <none> and password file <none>
I20260809 23:00:06.724519 3569 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:06.724651 3569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:06.724987 3569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:06.727910 3569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "237f5c40a918435a8c6104b0fec88c0c"
format_stamp: "Formatted at 2026-08-09 23:00:06 on dist-test-slave-vxj2"
server_key: "555d3e9c972d93695bd50331849b325b"
server_key_iv: "4f237ae661775667f37124dd57bf983d"
server_key_version: "encryptionkey@0"
I20260809 23:00:06.728799 3569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "237f5c40a918435a8c6104b0fec88c0c"
format_stamp: "Formatted at 2026-08-09 23:00:06 on dist-test-slave-vxj2"
server_key: "555d3e9c972d93695bd50331849b325b"
server_key_iv: "4f237ae661775667f37124dd57bf983d"
server_key_version: "encryptionkey@0"
I20260809 23:00:06.734201 3569 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 23:00:06.737586 3584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:06.738979 3569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 23:00:06.739182 3569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "237f5c40a918435a8c6104b0fec88c0c"
format_stamp: "Formatted at 2026-08-09 23:00:06 on dist-test-slave-vxj2"
server_key: "555d3e9c972d93695bd50331849b325b"
server_key_iv: "4f237ae661775667f37124dd57bf983d"
server_key_version: "encryptionkey@0"
I20260809 23:00:06.739431 3569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:06.745114 3569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:06.746076 3569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:06.746368 3569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:06.747332 3569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:00:06.748766 3569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:00:06.748893 3569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:06.749040 3569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:00:06.749128 3569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:06.782852 3569 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:43147
I20260809 23:00:06.782898 3714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:43147 every 8 connection(s)
I20260809 23:00:06.784190 3569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:00:06.787520 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 3569
I20260809 23:00:06.787774 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:00:06.788355 24703 external_mini_cluster.cc:1428] Setting key 7f7714b6bd07b94371ff291baeb11871
I20260809 23:00:06.791886 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44905
--builtin_ntp_servers=127.24.31.212:42219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 23:00:06.793771 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:06.786747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:57871 (local address 127.24.31.254:44905)
0809 23:00:06.786963 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:06.786968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:06.788733 (+ 1765us) server_negotiation.cc:407] Connection header received
0809 23:00:06.791071 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:06.791077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:06.791137 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:06.791237 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:06.792806 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:06.792813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:06.792818 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:06.792838 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:06.792877 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:06.792893 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:06.793006 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:06.793316 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 23:00:06.793391 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 23:00:06.795578 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:06.787334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44905 (local address 127.24.31.194:57871)
0809 23:00:06.788561 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:06.788606 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:06.790772 (+ 2166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:06.791499 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:06.791514 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:06.791640 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:06.792457 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:00:06.792462 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:06.792513 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:00:06.792625 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:00:06.793003 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:00:06.793016 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:00:06.793028 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 23:00:06.794134 (+ 1106us) client_negotiation.cc:240] Negotiation successful
0809 23:00:06.794405 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":385,"threads_started":1}
I20260809 23:00:06.796916 3715 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44905
I20260809 23:00:06.797245 3715 heartbeater.cc:464] Registering TS with master...
I20260809 23:00:06.797927 3715 heartbeater.cc:511] Master 127.24.31.254:44905 requested a full tablet report, sending...
I20260809 23:00:06.799506 3347 ts_manager.cc:194] Registered new tserver with Master: 237f5c40a918435a8c6104b0fec88c0c (127.24.31.194:43147)
W20260809 23:00:07.124951 3718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:07.125253 3718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:07.125347 3718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:07.132819 3718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:07.132918 3718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:07.133029 3718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:00:07.142225 3718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:07.143648 3718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:07.145547 3718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:07.159222 3723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:07.159317 3727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:07.159261 3725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:07.160300 3718 server_base.cc:1034] running on GCE node
I20260809 23:00:07.160938 3718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:07.161942 3718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:07.163172 3718 hybrid_clock.cc:648] HybridClock initialized: now 1786316407163110 us; error 60 us; skew 500 ppm
I20260809 23:00:07.163501 3718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:00:07.165939 3718 webserver.cc:460] Webserver started at http://127.24.31.195:41435/ using document root <none> and password file <none>
I20260809 23:00:07.166740 3718 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:07.166831 3718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:07.167091 3718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:07.169924 3718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "2560798be9764afbb19d11cbf8cc5a05"
format_stamp: "Formatted at 2026-08-09 23:00:07 on dist-test-slave-vxj2"
server_key: "96a6beb91f90eaebf679d722f2100fe2"
server_key_iv: "e75502cfa1f71bb55a405f263b4af656"
server_key_version: "encryptionkey@0"
I20260809 23:00:07.170768 3718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "2560798be9764afbb19d11cbf8cc5a05"
format_stamp: "Formatted at 2026-08-09 23:00:07 on dist-test-slave-vxj2"
server_key: "96a6beb91f90eaebf679d722f2100fe2"
server_key_iv: "e75502cfa1f71bb55a405f263b4af656"
server_key_version: "encryptionkey@0"
I20260809 23:00:07.175752 3718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:00:07.179066 3734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:07.180388 3718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:00:07.180629 3718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "2560798be9764afbb19d11cbf8cc5a05"
format_stamp: "Formatted at 2026-08-09 23:00:07 on dist-test-slave-vxj2"
server_key: "96a6beb91f90eaebf679d722f2100fe2"
server_key_iv: "e75502cfa1f71bb55a405f263b4af656"
server_key_version: "encryptionkey@0"
I20260809 23:00:07.181000 3718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:07.186736 3718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:07.187551 3718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:07.187851 3718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:07.188792 3718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:00:07.190229 3718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:00:07.190358 3718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:07.190496 3718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:00:07.190584 3718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:07.226408 3718 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:35513
I20260809 23:00:07.226456 3857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:35513 every 8 connection(s)
I20260809 23:00:07.227883 3718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:00:07.232043 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 3718
I20260809 23:00:07.232254 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:00:07.232887 24703 external_mini_cluster.cc:1428] Setting key bc8c949335bac0c1dc53fd08d83a25c8
I20260809 23:00:07.236836 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:07.230648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:47885 (local address 127.24.31.254:44905)
0809 23:00:07.230895 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:07.230900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:07.232607 (+ 1707us) server_negotiation.cc:407] Connection header received
0809 23:00:07.234701 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:07.234707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:07.234776 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:07.234883 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:07.236241 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:07.236245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:07.236249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:07.236265 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:07.236297 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:07.236309 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:07.236399 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:07.236576 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 23:00:07.236623 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 23:00:07.237879 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:07.231155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44905 (local address 127.24.31.195:47885)
0809 23:00:07.232410 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:07.232477 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:07.234372 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:07.235133 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:07.235141 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:07.235229 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:07.235972 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:00:07.235975 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:07.236012 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:00:07.236094 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:00:07.236395 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:00:07.236402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:00:07.236408 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:00:07.236621 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 23:00:07.236854 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":374,"threads_started":1}
I20260809 23:00:07.239193 3858 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44905
I20260809 23:00:07.239496 3858 heartbeater.cc:464] Registering TS with master...
I20260809 23:00:07.240084 3858 heartbeater.cc:511] Master 127.24.31.254:44905 requested a full tablet report, sending...
I20260809 23:00:07.241472 3347 ts_manager.cc:194] Registered new tserver with Master: 2560798be9764afbb19d11cbf8cc5a05 (127.24.31.195:35513)
I20260809 23:00:07.248152 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 23:00:07.254148 3567 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:00:07.252721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49140 (local address 127.24.31.254:44905)
0809 23:00:07.252895 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:07.252899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:07.253420 (+ 521us) server_negotiation.cc:407] Connection header received
0809 23:00:07.253552 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:07.253556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:07.253606 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:07.253684 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:07.253964 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49140: BlockingRecv error: recv got EOF from 127.0.0.1:49140 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:00:07.254745 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 3416
I20260809 23:00:07.258363 3872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:07.255758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45236 (local address 127.24.31.193:42615)
0809 23:00:07.256298 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:07.256306 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:07.256331 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:00:07.256397 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:07.256403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:07.256656 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:07.256839 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:07.257743 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:07.257749 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:07.257759 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:07.257777 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:07.257814 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:07.257828 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:07.257930 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:07.258051 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 23:00:07.258131 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":275,"threads_started":1}
I20260809 23:00:07.259636 3560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:07.341430 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 3416
I20260809 23:00:07.361886 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 3569
I20260809 23:00:07.365453 3875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:07.362837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34700 (local address 127.24.31.194:43147)
0809 23:00:07.363397 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:07.363404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:07.363432 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:00:07.363505 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:07.363511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:07.363725 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:07.363851 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:07.364725 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:07.364731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:07.364742 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:07.364762 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:07.364806 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:07.364820 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:07.364981 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:07.365058 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 23:00:07.365137 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":282,"threads_started":1}
I20260809 23:00:07.366753 3709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:07.454524 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 3569
I20260809 23:00:07.473760 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 3718
I20260809 23:00:07.478019 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:07.474846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52580 (local address 127.24.31.195:35513)
0809 23:00:07.475501 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:07.475508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:07.475539 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:00:07.475600 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:07.475606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:07.475826 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:07.476020 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:07.477252 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:07.477259 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:07.477270 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:00:07.477290 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:07.477333 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:00:07.477347 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:07.477532 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:07.477611 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 23:00:07.477713 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":344,"threads_started":1}
I20260809 23:00:07.479249 3853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:07.561439 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 3718
I20260809 23:00:07.583520 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 3312
I20260809 23:00:07.584858 3387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:07.691013 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 3312
2026-08-09T23:00:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2408 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:00:07 dist-test-slave-vxj2 krb5kdc[3882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:00:07 dist-test-slave-vxj2 krb5kdc[3882](info): set up 2 sockets
Aug 09 23:00:07 dist-test-slave-vxj2 krb5kdc[3882](info): commencing operation
krb5kdc: starting...
W20260809 23:00:08.878965 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.005s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:00:09 dist-test-slave-vxj2 krb5kdc[3882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:00:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:00:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:00:09.136768 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:38559
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:38559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:00:09.470654 3896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:09.471026 3896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:09.471194 3896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:09.478287 3896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:00:09.478441 3896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:09.478545 3896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:09.478647 3896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:00:09.478729 3896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:00:09.488454 3896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:38559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:38559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:09.489918 3896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:09.491276 3896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:09.503371 3902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:09.503403 3903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:09.503252 3905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:09.504674 3896 server_base.cc:1034] running on GCE node
I20260809 23:00:09.505514 3896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:09.506989 3896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:09.508226 3896 hybrid_clock.cc:648] HybridClock initialized: now 1786316409508186 us; error 53 us; skew 500 ppm
I20260809 23:00:09.508577 3896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:00:09 dist-test-slave-vxj2 krb5kdc[3882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:00:09.515513 3896 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:00:09.517215 3896 webserver.cc:460] Webserver started at http://127.24.31.254:40903/ using document root <none> and password file <none>
I20260809 23:00:09.517900 3896 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:09.518013 3896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:09.518280 3896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:09.521427 3896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "f38dc860d8f146368e8f5b6e85ad6f8b"
format_stamp: "Formatted at 2026-08-09 23:00:09 on dist-test-slave-vxj2"
server_key: "e7dd26c8cc72ed0e2edbd7658fabfd2f"
server_key_iv: "e36ac1535c6a25ed641255bde09f6d6a"
server_key_version: "encryptionkey@0"
I20260809 23:00:09.522161 3896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "f38dc860d8f146368e8f5b6e85ad6f8b"
format_stamp: "Formatted at 2026-08-09 23:00:09 on dist-test-slave-vxj2"
server_key: "e7dd26c8cc72ed0e2edbd7658fabfd2f"
server_key_iv: "e36ac1535c6a25ed641255bde09f6d6a"
server_key_version: "encryptionkey@0"
I20260809 23:00:09.527442 3896 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 23:00:09.530884 3913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:09.532217 3896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:00:09.532354 3896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "f38dc860d8f146368e8f5b6e85ad6f8b"
format_stamp: "Formatted at 2026-08-09 23:00:09 on dist-test-slave-vxj2"
server_key: "e7dd26c8cc72ed0e2edbd7658fabfd2f"
server_key_iv: "e36ac1535c6a25ed641255bde09f6d6a"
server_key_version: "encryptionkey@0"
I20260809 23:00:09.532519 3896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:09.554260 3896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:09.558593 3896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:09.558844 3896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:09.578532 3896 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:38559
I20260809 23:00:09.578567 3970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:38559 every 8 connection(s)
I20260809 23:00:09.579924 3896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:00:09.581216 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 3896
I20260809 23:00:09.581480 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:00:09.582295 24703 external_mini_cluster.cc:1428] Setting key cdf70ce2e658c72404f1fd4fa581d705
I20260809 23:00:09.584710 3971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:00:09.598598 3971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b: Bootstrap starting.
Aug 09 23:00:09 dist-test-slave-vxj2 krb5kdc[3882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:00:09.602636 3971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b: Neither blocks nor log segments found. Creating new log.
I20260809 23:00:09.604031 3971 log.cc:824] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:00:09.607686 3971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b: No bootstrap required, opened a new log
I20260809 23:00:09.608634 3974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:09.584685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.24.31.254:38559)
0809 23:00:09.586110 (+ 1425us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:09.586127 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:09.586177 (+ 50us) server_negotiation.cc:407] Connection header received
0809 23:00:09.588194 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:09.588223 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:09.588746 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:09.589215 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:09.590638 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:09.592748 (+ 2110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:09.594205 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:09.595272 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:09.595495 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:09.599718 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:09.599728 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:09.599747 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:09.599797 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:09.603820 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:09.604594 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:09.604606 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:09.604619 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:09.604842 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:09.605408 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:09.605413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:09.605415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:09.606133 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:09.606487 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:09.606786 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 23:00:09.607221 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1157,"thread_start_us":882,"threads_started":1}
I20260809 23:00:09.616369 3971 raft_consensus.cc:348] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38dc860d8f146368e8f5b6e85ad6f8b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38559 } }
I20260809 23:00:09.616750 3971 raft_consensus.cc:374] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:00:09.616870 3971 raft_consensus.cc:729] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f38dc860d8f146368e8f5b6e85ad6f8b, State: Initialized, Role: FOLLOWER
I20260809 23:00:09.617532 3971 consensus_queue.cc:260] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38dc860d8f146368e8f5b6e85ad6f8b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38559 } }
I20260809 23:00:09.617774 3971 raft_consensus.cc:388] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:00:09.617909 3971 raft_consensus.cc:482] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:00:09.618088 3971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:00:09.620618 3971 raft_consensus.cc:504] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38dc860d8f146368e8f5b6e85ad6f8b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38559 } }
I20260809 23:00:09.621289 3971 leader_election.cc:302] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f38dc860d8f146368e8f5b6e85ad6f8b; no voters:
I20260809 23:00:09.621917 3971 leader_election.cc:288] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:00:09.622072 3976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:00:09.623170 3976 raft_consensus.cc:686] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [term 1 LEADER]: Becoming Leader. State: Replica: f38dc860d8f146368e8f5b6e85ad6f8b, State: Running, Role: LEADER
I20260809 23:00:09.623728 3976 consensus_queue.cc:237] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38dc860d8f146368e8f5b6e85ad6f8b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38559 } }
I20260809 23:00:09.623920 3971 sys_catalog.cc:564] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:00:09.626619 3977 sys_catalog.cc:455] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f38dc860d8f146368e8f5b6e85ad6f8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38dc860d8f146368e8f5b6e85ad6f8b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38559 } } }
I20260809 23:00:09.626998 3977 sys_catalog.cc:458] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [sys.catalog]: This master's current role is: LEADER
I20260809 23:00:09.626711 3978 sys_catalog.cc:455] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f38dc860d8f146368e8f5b6e85ad6f8b. Latest consensus state: current_term: 1 leader_uuid: "f38dc860d8f146368e8f5b6e85ad6f8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f38dc860d8f146368e8f5b6e85ad6f8b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38559 } } }
I20260809 23:00:09.627163 3978 sys_catalog.cc:458] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b [sys.catalog]: This master's current role is: LEADER
I20260809 23:00:09.632102 3985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:00:09.637276 3985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:00:09.645766 3985 catalog_manager.cc:1347] Generated new cluster ID: f9dd6d7bdd3c4969a1f101d3553fb7b9
I20260809 23:00:09.645895 3985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:00:09.670059 3985 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:00:09.671983 3985 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:00:09.680649 3985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b: Generated new TSK 0
I20260809 23:00:09.681717 3985 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 23:00:09.750923 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:38559
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:00:10.083761 4000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:10.084326 4000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:10.084527 4000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:10.092473 4000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:10.092669 4000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:10.092895 4000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:00:10.103287 4000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:38559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:10.104751 4000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:10.106549 4000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:10.120898 4005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:10.120968 4006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:10.120968 4009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:10.121800 4000 server_base.cc:1034] running on GCE node
I20260809 23:00:10.122493 4000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:10.123533 4000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:10.124804 4000 hybrid_clock.cc:648] HybridClock initialized: now 1786316410124754 us; error 51 us; skew 500 ppm
I20260809 23:00:10.125142 4000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:00:10 dist-test-slave-vxj2 krb5kdc[3882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:00:10.128672 4000 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:00:10.130319 4000 webserver.cc:460] Webserver started at http://127.24.31.193:39217/ using document root <none> and password file <none>
I20260809 23:00:10.131002 4000 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:10.131143 4000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:10.131436 4000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:10.134416 4000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "edf1ca3af72e4c458e15db56b5881e51"
format_stamp: "Formatted at 2026-08-09 23:00:10 on dist-test-slave-vxj2"
server_key: "1b8b91f6fc2e889200c9492de629ee1b"
server_key_iv: "3b8744f1dafce93d3102e01552230f2f"
server_key_version: "encryptionkey@0"
I20260809 23:00:10.135161 4000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "edf1ca3af72e4c458e15db56b5881e51"
format_stamp: "Formatted at 2026-08-09 23:00:10 on dist-test-slave-vxj2"
server_key: "1b8b91f6fc2e889200c9492de629ee1b"
server_key_iv: "3b8744f1dafce93d3102e01552230f2f"
server_key_version: "encryptionkey@0"
I20260809 23:00:10.140298 4000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 23:00:10.143647 4016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:10.145025 4000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:00:10.145246 4000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "edf1ca3af72e4c458e15db56b5881e51"
format_stamp: "Formatted at 2026-08-09 23:00:10 on dist-test-slave-vxj2"
server_key: "1b8b91f6fc2e889200c9492de629ee1b"
server_key_iv: "3b8744f1dafce93d3102e01552230f2f"
server_key_version: "encryptionkey@0"
I20260809 23:00:10.145449 4000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:10.165416 4000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:10.169705 4000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:10.170010 4000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:10.171247 4000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:00:10.172729 4000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:00:10.172858 4000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:10.173015 4000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:00:10.173102 4000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:10.208735 4000 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:45769
I20260809 23:00:10.208768 4144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:45769 every 8 connection(s)
I20260809 23:00:10.210196 4000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:00:10.218750 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 4000
I20260809 23:00:10.218976 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:00:10.219682 24703 external_mini_cluster.cc:1428] Setting key 31a1bbdcd604a2b82ae36307cc03c431
Aug 09 23:00:10 dist-test-slave-vxj2 krb5kdc[3882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:00:10.232286 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:10.212831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:60535 (local address 127.24.31.254:38559)
0809 23:00:10.213365 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:10.213370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:10.214387 (+ 1017us) server_negotiation.cc:407] Connection header received
0809 23:00:10.216276 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:10.216281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:10.216373 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:10.216494 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:10.218071 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:10.219173 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:10.221087 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:10.221728 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:10.221837 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:10.226226 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:10.226232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:10.226239 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:10.226277 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:10.228906 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:10.229836 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:10.229846 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:10.229848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:10.229974 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:10.230433 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:10.230440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:10.230451 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:10.230831 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:10.230997 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:10.231668 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 23:00:10.231909 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":241,"threads_started":1}
I20260809 23:00:10.233286 4148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:10.213488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:38559 (local address 127.24.31.193:60535)
0809 23:00:10.214229 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:10.214269 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:10.216006 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:10.216732 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:10.216740 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:10.217310 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:10.217877 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:10.217912 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:10.219394 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:10.219399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:10.220914 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:10.220928 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:10.221991 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:10.222000 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:10.222318 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:10.223046 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:10.223087 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:10.226010 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:00:10.229057 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:00:10.229065 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:00:10.229080 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:00:10.229690 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:00:10.230090 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:00:10.230095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:00:10.230098 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:00:10.230291 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:00:10.230975 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:00:10.230982 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:00:10.231305 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 23:00:10.231557 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 23:00:10.232099 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":294,"threads_started":1}
I20260809 23:00:10.234957 4145 heartbeater.cc:347] Connected to a master server at 127.24.31.254:38559
I20260809 23:00:10.235271 4145 heartbeater.cc:464] Registering TS with master...
I20260809 23:00:10.236194 4145 heartbeater.cc:511] Master 127.24.31.254:38559 requested a full tablet report, sending...
I20260809 23:00:10.238652 3932 ts_manager.cc:194] Registered new tserver with Master: edf1ca3af72e4c458e15db56b5881e51 (127.24.31.193:45769)
I20260809 23:00:10.240993 3932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:60535
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 23:00:10.282423 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:38559
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:00:10.613461 4151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:10.613742 4151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:10.613906 4151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:10.621353 4151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:10.621472 4151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:10.621634 4151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:00:10.631784 4151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:38559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:10.633206 4151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:10.634550 4151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:10.647892 4156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:10.647836 4157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:10.647871 4160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:10.649087 4151 server_base.cc:1034] running on GCE node
I20260809 23:00:10.649729 4151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:10.650727 4151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:10.651923 4151 hybrid_clock.cc:648] HybridClock initialized: now 1786316410651888 us; error 24 us; skew 500 ppm
I20260809 23:00:10.652200 4151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:00:10 dist-test-slave-vxj2 krb5kdc[3882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:00:10.655416 4151 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:00:10.656833 4151 webserver.cc:460] Webserver started at http://127.24.31.194:44733/ using document root <none> and password file <none>
I20260809 23:00:10.657567 4151 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:10.657670 4151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:10.657912 4151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:10.660892 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "e2d19f6aa18d436c943668bc4ec3a643"
format_stamp: "Formatted at 2026-08-09 23:00:10 on dist-test-slave-vxj2"
server_key: "c18de641a1f6fd6791ded19d803a37c1"
server_key_iv: "be3753b8c793b5ffa4ae109ff098e6a9"
server_key_version: "encryptionkey@0"
I20260809 23:00:10.661742 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "e2d19f6aa18d436c943668bc4ec3a643"
format_stamp: "Formatted at 2026-08-09 23:00:10 on dist-test-slave-vxj2"
server_key: "c18de641a1f6fd6791ded19d803a37c1"
server_key_iv: "be3753b8c793b5ffa4ae109ff098e6a9"
server_key_version: "encryptionkey@0"
I20260809 23:00:10.666872 4151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:00:10.670245 4169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:10.671506 4151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 23:00:10.671638 4151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "e2d19f6aa18d436c943668bc4ec3a643"
format_stamp: "Formatted at 2026-08-09 23:00:10 on dist-test-slave-vxj2"
server_key: "c18de641a1f6fd6791ded19d803a37c1"
server_key_iv: "be3753b8c793b5ffa4ae109ff098e6a9"
server_key_version: "encryptionkey@0"
I20260809 23:00:10.671795 4151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:10.693041 4151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:10.697599 4151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:10.697863 4151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:10.698834 4151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:00:10.700179 4151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:00:10.700258 4151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:10.700340 4151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:00:10.700384 4151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:10.735082 4151 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:42743
I20260809 23:00:10.735246 4305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:42743 every 8 connection(s)
I20260809 23:00:10.736654 4151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:00:10.738399 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 4151
I20260809 23:00:10.738618 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:00:10.739104 24703 external_mini_cluster.cc:1428] Setting key eba7cc6b8bdcd74dbbf4fbb7aa101deb
Aug 09 23:00:10 dist-test-slave-vxj2 krb5kdc[3882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316410, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:00:10.760010 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:10.739317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:49063 (local address 127.24.31.254:38559)
0809 23:00:10.739974 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:10.739979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:10.741057 (+ 1078us) server_negotiation.cc:407] Connection header received
0809 23:00:10.745338 (+ 4281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:10.745343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:10.745414 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:10.745550 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:10.747181 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:10.747927 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:10.749698 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:10.750200 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:10.750343 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:10.754106 (+ 3763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:10.754112 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:10.754115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:10.754148 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:10.756753 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:10.757605 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:10.757609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:10.757617 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:10.757753 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:10.758203 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:10.758212 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:10.758214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:10.758485 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:10.758646 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:10.759375 (+ 729us) server_negotiation.cc:299] Negotiation successful
0809 23:00:10.759624 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":410,"threads_started":1}
I20260809 23:00:10.761090 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:10.740025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:38559 (local address 127.24.31.194:49063)
0809 23:00:10.740894 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:10.740940 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:10.742602 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:10.745865 (+ 3263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:10.745875 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:10.746410 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:10.746997 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:10.747015 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:10.748097 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:10.748101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:10.749550 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:10.749563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:10.750435 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:10.750444 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:10.750695 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:10.751405 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:10.751432 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:10.753874 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:00:10.756928 (+ 3054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:00:10.756938 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:00:10.756955 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:00:10.757473 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:00:10.757875 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:00:10.757880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:00:10.757884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:00:10.758079 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:00:10.758618 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:00:10.758627 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:00:10.759001 (+ 374us) client_negotiation.cc:769] Sending connection context
0809 23:00:10.759270 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 23:00:10.759838 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":363,"threads_started":1}
I20260809 23:00:10.762717 4306 heartbeater.cc:347] Connected to a master server at 127.24.31.254:38559
I20260809 23:00:10.763031 4306 heartbeater.cc:464] Registering TS with master...
I20260809 23:00:10.763936 4306 heartbeater.cc:511] Master 127.24.31.254:38559 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
I20260809 23:00:10.765551 3932 ts_manager.cc:194] Registered new tserver with Master: e2d19f6aa18d436c943668bc4ec3a643 (127.24.31.194:42743)
I20260809 23:00:10.766353 3932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:49063
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 23:00:10.796697 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:38559
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:00:11.132812 4313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:11.133088 4313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:11.133279 4313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:11.140846 4313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:11.140955 4313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:11.141070 4313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:00:11.151746 4313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:38559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:11.153076 4313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:11.154389 4313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:11.167647 4321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:11.167732 4319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:11.167685 4318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:11.168953 4313 server_base.cc:1034] running on GCE node
I20260809 23:00:11.169638 4313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:11.170645 4313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:11.171885 4313 hybrid_clock.cc:648] HybridClock initialized: now 1786316411171852 us; error 42 us; skew 500 ppm
I20260809 23:00:11.172220 4313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:00:11 dist-test-slave-vxj2 krb5kdc[3882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:00:11.175580 4313 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:00:11.177202 4313 webserver.cc:460] Webserver started at http://127.24.31.195:32889/ using document root <none> and password file <none>
I20260809 23:00:11.177970 4313 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:11.178077 4313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:11.178402 4313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:00:11.181494 4313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "cb59e5af09b64e9490c9f7225b8116df"
format_stamp: "Formatted at 2026-08-09 23:00:11 on dist-test-slave-vxj2"
server_key: "ea886449441ac93478b77d7bdca18915"
server_key_iv: "e34a39f53081abca602abf774721d493"
server_key_version: "encryptionkey@0"
I20260809 23:00:11.182284 4313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "cb59e5af09b64e9490c9f7225b8116df"
format_stamp: "Formatted at 2026-08-09 23:00:11 on dist-test-slave-vxj2"
server_key: "ea886449441ac93478b77d7bdca18915"
server_key_iv: "e34a39f53081abca602abf774721d493"
server_key_version: "encryptionkey@0"
I20260809 23:00:11.187851 4313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 23:00:11.191594 4331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:11.192970 4313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 23:00:11.193231 4313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "cb59e5af09b64e9490c9f7225b8116df"
format_stamp: "Formatted at 2026-08-09 23:00:11 on dist-test-slave-vxj2"
server_key: "ea886449441ac93478b77d7bdca18915"
server_key_iv: "e34a39f53081abca602abf774721d493"
server_key_version: "encryptionkey@0"
I20260809 23:00:11.193467 4313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:11.227643 4313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:11.232688 4313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:11.233019 4313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:11.234081 4313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:00:11.235471 4313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:00:11.235545 4313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:11.235726 4313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:00:11.235798 4313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:11.245016 4145 heartbeater.cc:503] Master 127.24.31.254:38559 was elected leader, sending a full tablet report...
I20260809 23:00:11.269235 4313 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:36889
I20260809 23:00:11.269273 4465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:36889 every 8 connection(s)
I20260809 23:00:11.270573 4313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:00:11.273474 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 4313
I20260809 23:00:11.273695 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:00:11.274279 24703 external_mini_cluster.cc:1428] Setting key c0a24e636e30e31e529d5751f68ba33f
Aug 09 23:00:11 dist-test-slave-vxj2 krb5kdc[3882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:00:11.292083 4470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.273322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:37697 (local address 127.24.31.254:38559)
0809 23:00:11.273861 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.273870 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.274846 (+ 976us) server_negotiation.cc:407] Connection header received
0809 23:00:11.277570 (+ 2724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.277574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.277676 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.277798 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:11.279300 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.280071 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.281709 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.282104 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.282198 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.285686 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:11.285690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:11.285694 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:11.285727 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:11.289020 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:11.289937 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:11.289947 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:11.289950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:11.290097 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:11.290602 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:11.290606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:11.290608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:11.290870 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:11.290996 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.291643 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.291807 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":239,"threads_started":1}
I20260809 23:00:11.293102 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.273834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:38559 (local address 127.24.31.195:37697)
0809 23:00:11.274643 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:11.274695 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:11.277244 (+ 2549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:11.278016 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:11.278025 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:11.278575 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:00:11.279153 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.279170 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.280215 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.280219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.281580 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.281593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.282328 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.282333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.282527 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.283166 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:00:11.283194 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:00:11.285490 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:00:11.289189 (+ 3699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:00:11.289197 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:00:11.289213 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:00:11.289781 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:00:11.290225 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:00:11.290229 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:00:11.290232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:00:11.290481 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:00:11.290988 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:00:11.290994 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:00:11.291319 (+ 325us) client_negotiation.cc:769] Sending connection context
0809 23:00:11.291544 (+ 225us) client_negotiation.cc:240] Negotiation successful
0809 23:00:11.291925 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":310,"threads_started":1}
I20260809 23:00:11.294540 4467 heartbeater.cc:347] Connected to a master server at 127.24.31.254:38559
I20260809 23:00:11.294881 4467 heartbeater.cc:464] Registering TS with master...
I20260809 23:00:11.295643 4467 heartbeater.cc:511] Master 127.24.31.254:38559 requested a full tablet report, sending...
I20260809 23:00:11.297055 3932 ts_manager.cc:194] Registered new tserver with Master: cb59e5af09b64e9490c9f7225b8116df (127.24.31.195:36889)
I20260809 23:00:11.298013 3932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:37697
I20260809 23:00:11.302891 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:00:11.320587 4470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.307610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.24.31.254:38559)
0809 23:00:11.307814 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.307819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.308303 (+ 484us) server_negotiation.cc:407] Connection header received
0809 23:00:11.308434 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.308438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.308509 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.308595 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:11.310130 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.310927 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.312121 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.312510 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.312602 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.314992 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:11.314997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:11.315000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:11.315036 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:11.317926 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:11.318672 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:11.318677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:11.318680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:11.318844 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:11.319239 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:11.319243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:11.319246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:11.319627 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:11.319838 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.320141 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.320346 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 23:00:11.325240 3932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:00:11.332468 3932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:00:11.358086 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.345613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:36889 (local address 127.0.0.1:54682)
0809 23:00:11.346916 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:11.346962 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:11.347140 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:11.350529 (+ 3389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:11.350534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:11.350561 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:00:11.351049 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.351066 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.353198 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.353203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.355436 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.355450 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.356878 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.356886 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.357111 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.357202 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 23:00:11.357345 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 23:00:11.357577 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":121,"thread_start_us":876,"threads_started":1}
I20260809 23:00:11.358731 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.346202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:45769 (local address 127.0.0.1:46470)
0809 23:00:11.347814 (+ 1612us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:11.347833 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:11.347944 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:11.349511 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:11.349519 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:11.349572 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:00:11.350173 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.350188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.352476 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.352482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.355511 (+ 3029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.355527 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.357524 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.357531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.357697 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.357743 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 23:00:11.357819 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 23:00:11.357955 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1455,"mutex_wait_us":4,"thread_start_us":1293,"threads_started":1}
I20260809 23:00:11.359194 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.347011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.24.31.195:36889)
0809 23:00:11.349915 (+ 2904us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.349924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.349959 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:00:11.350059 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.350069 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.350322 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.350542 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:00:11.351330 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.353065 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.355604 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.356649 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.356783 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.358089 (+ 1306us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.358165 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.358631 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1790,"thread_start_us":1621,"threads_started":1}
I20260809 23:00:11.359231 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.347873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:42743 (local address 127.0.0.1:47342)
0809 23:00:11.349809 (+ 1936us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:11.349833 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:11.349957 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:11.350856 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:11.350862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:11.350896 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:00:11.351386 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.351395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.353042 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.353048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.356227 (+ 3179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.356243 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.358388 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.358393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.358561 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.358600 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 23:00:11.358706 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 23:00:11.358842 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1804,"mutex_wait_us":252,"thread_start_us":1615,"threads_started":1}
I20260809 23:00:11.359360 4487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.348418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47342 (local address 127.24.31.194:42743)
0809 23:00:11.349323 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.349335 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.350102 (+ 767us) server_negotiation.cc:407] Connection header received
0809 23:00:11.350230 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.350238 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.350464 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.350671 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:00:11.351561 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.352856 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.356404 (+ 3548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.357875 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.358036 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.358241 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.358824 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.359028 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":546,"threads_started":1}
I20260809 23:00:11.359979 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.347584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46470 (local address 127.24.31.193:45769)
0809 23:00:11.348613 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.348626 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.348730 (+ 104us) server_negotiation.cc:407] Connection header received
0809 23:00:11.348860 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.348871 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.349145 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.350276 (+ 1131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:00:11.350566 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.352314 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.355719 (+ 3405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.357256 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.359094 (+ 1838us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.359354 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.359442 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.359633 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":589,"threads_started":1}
I20260809 23:00:11.363406 4224 tablet_service.cc:1467] Processing CreateTablet for tablet da58e6dbacfa47fa8288076312c321d6 (DEFAULT_TABLE table=test-table [id=66cc009f949d4853ae56eaa480e40ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:00:11.363405 4384 tablet_service.cc:1467] Processing CreateTablet for tablet da58e6dbacfa47fa8288076312c321d6 (DEFAULT_TABLE table=test-table [id=66cc009f949d4853ae56eaa480e40ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:00:11.363405 4068 tablet_service.cc:1467] Processing CreateTablet for tablet da58e6dbacfa47fa8288076312c321d6 (DEFAULT_TABLE table=test-table [id=66cc009f949d4853ae56eaa480e40ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:00:11.364770 4384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da58e6dbacfa47fa8288076312c321d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:00:11.364770 4068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da58e6dbacfa47fa8288076312c321d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:00:11.364768 4224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da58e6dbacfa47fa8288076312c321d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:00:11.377570 4489 tablet_bootstrap.cc:492] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: Bootstrap starting.
I20260809 23:00:11.381419 4489 tablet_bootstrap.cc:654] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: Neither blocks nor log segments found. Creating new log.
I20260809 23:00:11.383122 4489 log.cc:824] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: Log is configured to *not* fsync() on all Append() calls
I20260809 23:00:11.388096 4491 tablet_bootstrap.cc:492] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: Bootstrap starting.
I20260809 23:00:11.389511 4490 tablet_bootstrap.cc:492] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: Bootstrap starting.
I20260809 23:00:11.391918 4489 tablet_bootstrap.cc:492] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: No bootstrap required, opened a new log
I20260809 23:00:11.392238 4489 ts_tablet_manager.cc:1397] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.010s
I20260809 23:00:11.392889 4491 tablet_bootstrap.cc:654] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: Neither blocks nor log segments found. Creating new log.
I20260809 23:00:11.393515 4490 tablet_bootstrap.cc:654] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: Neither blocks nor log segments found. Creating new log.
I20260809 23:00:11.394426 4491 log.cc:824] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: Log is configured to *not* fsync() on all Append() calls
I20260809 23:00:11.394881 4490 log.cc:824] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: Log is configured to *not* fsync() on all Append() calls
I20260809 23:00:11.400362 4489 raft_consensus.cc:348] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.400770 4489 raft_consensus.cc:374] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:00:11.400874 4489 raft_consensus.cc:729] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2d19f6aa18d436c943668bc4ec3a643, State: Initialized, Role: FOLLOWER
I20260809 23:00:11.401690 4489 consensus_queue.cc:260] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.402969 4306 heartbeater.cc:503] Master 127.24.31.254:38559 was elected leader, sending a full tablet report...
I20260809 23:00:11.403632 4489 ts_tablet_manager.cc:1428] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 23:00:11.404101 4491 tablet_bootstrap.cc:492] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: No bootstrap required, opened a new log
I20260809 23:00:11.404124 4490 tablet_bootstrap.cc:492] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: No bootstrap required, opened a new log
I20260809 23:00:11.404400 4490 ts_tablet_manager.cc:1397] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.005s
I20260809 23:00:11.404570 4491 ts_tablet_manager.cc:1397] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20260809 23:00:11.412665 4490 raft_consensus.cc:348] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.413080 4490 raft_consensus.cc:374] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:00:11.413228 4490 raft_consensus.cc:729] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb59e5af09b64e9490c9f7225b8116df, State: Initialized, Role: FOLLOWER
I20260809 23:00:11.413803 4490 consensus_queue.cc:260] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.414507 4491 raft_consensus.cc:348] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.414976 4467 heartbeater.cc:503] Master 127.24.31.254:38559 was elected leader, sending a full tablet report...
I20260809 23:00:11.415144 4491 raft_consensus.cc:374] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:00:11.415282 4491 raft_consensus.cc:729] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf1ca3af72e4c458e15db56b5881e51, State: Initialized, Role: FOLLOWER
I20260809 23:00:11.415560 4490 ts_tablet_manager.cc:1428] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260809 23:00:11.415915 4491 consensus_queue.cc:260] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.417552 4491 ts_tablet_manager.cc:1428] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260809 23:00:11.462967 4146 tablet.cc:2369] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:00:11.488973 4307 tablet.cc:2369] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:00:11.522275 4468 tablet.cc:2369] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:00:11.618957 4496 raft_consensus.cc:482] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:00:11.619319 4496 raft_consensus.cc:504] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.621009 4496 leader_election.cc:288] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edf1ca3af72e4c458e15db56b5881e51 (127.24.31.193:45769), e2d19f6aa18d436c943668bc4ec3a643 (127.24.31.194:42743)
I20260809 23:00:11.624974 4497 raft_consensus.cc:482] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:00:11.625332 4497 raft_consensus.cc:504] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.628664 4497 leader_election.cc:288] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb59e5af09b64e9490c9f7225b8116df (127.24.31.195:36889), e2d19f6aa18d436c943668bc4ec3a643 (127.24.31.194:42743)
I20260809 23:00:11.629292 4487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.622196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:39593 (local address 127.24.31.194:42743)
0809 23:00:11.622411 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.622419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.622922 (+ 503us) server_negotiation.cc:407] Connection header received
0809 23:00:11.623058 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.623066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.623144 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.623252 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:00:11.623877 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.624970 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.627103 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.628014 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.628165 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.628233 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.628772 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.628911 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 23:00:11.629647 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.621566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:45769 (local address 127.24.31.195:44381)
0809 23:00:11.621814 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:11.621835 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:11.622017 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:11.622519 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:11.622526 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:11.622562 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:00:11.622782 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.622795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.624291 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.624294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.626231 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.626247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.627975 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.627981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.628174 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.628207 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 23:00:11.629192 (+ 985us) client_negotiation.cc:240] Negotiation successful
0809 23:00:11.629378 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 23:00:11.630178 4498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.622018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:42743 (local address 127.24.31.195:39593)
0809 23:00:11.622788 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:11.622804 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:11.622926 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:11.623258 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:11.623261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:11.623279 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:00:11.623624 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.623632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.625192 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.625198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.626942 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.626955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.628394 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.628402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.628575 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.628610 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 23:00:11.629663 (+ 1053us) client_negotiation.cc:240] Negotiation successful
0809 23:00:11.629851 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":433,"threads_started":1}
I20260809 23:00:11.630200 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.621617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:44381 (local address 127.24.31.193:45769)
0809 23:00:11.621830 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.621838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.621931 (+ 93us) server_negotiation.cc:407] Connection header received
0809 23:00:11.622210 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.622218 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.622329 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.622451 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:00:11.622967 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.624099 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.626403 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.627778 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.627994 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.628104 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.628544 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.629971 (+ 1427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 23:00:11.631153 4090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da58e6dbacfa47fa8288076312c321d6" candidate_uuid: "cb59e5af09b64e9490c9f7225b8116df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf1ca3af72e4c458e15db56b5881e51" is_pre_election: true
I20260809 23:00:11.631590 4252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da58e6dbacfa47fa8288076312c321d6" candidate_uuid: "cb59e5af09b64e9490c9f7225b8116df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2d19f6aa18d436c943668bc4ec3a643" is_pre_election: true
I20260809 23:00:11.631701 4090 raft_consensus.cc:2454] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb59e5af09b64e9490c9f7225b8116df in term 0.
I20260809 23:00:11.632158 4252 raft_consensus.cc:2454] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb59e5af09b64e9490c9f7225b8116df in term 0.
I20260809 23:00:11.632671 4335 leader_election.cc:302] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb59e5af09b64e9490c9f7225b8116df, edf1ca3af72e4c458e15db56b5881e51; no voters:
I20260809 23:00:11.633168 4496 raft_consensus.cc:2781] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:00:11.633350 4496 raft_consensus.cc:482] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:00:11.633469 4496 raft_consensus.cc:3037] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:00:11.636860 4496 raft_consensus.cc:504] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.637607 4496 leader_election.cc:288] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [CANDIDATE]: Term 1 election: Requested vote from peers edf1ca3af72e4c458e15db56b5881e51 (127.24.31.193:45769), e2d19f6aa18d436c943668bc4ec3a643 (127.24.31.194:42743)
I20260809 23:00:11.638048 4090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da58e6dbacfa47fa8288076312c321d6" candidate_uuid: "cb59e5af09b64e9490c9f7225b8116df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf1ca3af72e4c458e15db56b5881e51"
I20260809 23:00:11.638062 4252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da58e6dbacfa47fa8288076312c321d6" candidate_uuid: "cb59e5af09b64e9490c9f7225b8116df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2d19f6aa18d436c943668bc4ec3a643"
I20260809 23:00:11.638262 4090 raft_consensus.cc:3037] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:00:11.638278 4252 raft_consensus.cc:3037] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:00:11.638872 4499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.627481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:36889 (local address 127.24.31.193:47955)
0809 23:00:11.631067 (+ 3586us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:11.631085 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:11.631236 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:11.631724 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:11.631727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:11.631747 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:00:11.632451 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.632464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.633945 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.633952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.635661 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.635692 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.637372 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.637380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.637562 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.637605 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 23:00:11.638258 (+ 653us) client_negotiation.cc:240] Negotiation successful
0809 23:00:11.638407 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2583,"mutex_wait_us":47,"thread_start_us":1628,"threads_started":1}
I20260809 23:00:11.640625 4090 raft_consensus.cc:2454] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb59e5af09b64e9490c9f7225b8116df in term 1.
I20260809 23:00:11.641341 4252 raft_consensus.cc:2454] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb59e5af09b64e9490c9f7225b8116df in term 1.
I20260809 23:00:11.638191 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.628261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:47955 (local address 127.24.31.195:36889)
0809 23:00:11.628458 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.628464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.631152 (+ 2688us) server_negotiation.cc:407] Connection header received
0809 23:00:11.631382 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.631386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.631448 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.631502 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:00:11.632767 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.633786 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.635830 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.637143 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.637323 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.637398 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.637825 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.637954 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 23:00:11.641646 4487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.628918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:48145 (local address 127.24.31.194:42743)
0809 23:00:11.629620 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.629625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.631068 (+ 1443us) server_negotiation.cc:407] Connection header received
0809 23:00:11.631361 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.631366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.631438 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.631502 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:00:11.632434 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.633593 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.636213 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.637524 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.637684 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.637763 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.641211 (+ 3448us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.641346 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":1524,"threads_started":1}
I20260809 23:00:11.641012 4335 leader_election.cc:302] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb59e5af09b64e9490c9f7225b8116df, edf1ca3af72e4c458e15db56b5881e51; no voters:
I20260809 23:00:11.641729 4500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.628864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:42743 (local address 127.24.31.193:48145)
0809 23:00:11.630976 (+ 2112us) negotiation.cc:107] Waiting for socket to connect
0809 23:00:11.630998 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 23:00:11.631184 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:00:11.631687 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:00:11.631692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:00:11.631720 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:00:11.632187 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.632201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.633781 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.633786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.635999 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:00:11.636014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.640821 (+ 4807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.640829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:00:11.641021 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.641062 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 23:00:11.641190 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 23:00:11.641346 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1981,"mutex_wait_us":27,"thread_start_us":1925,"threads_started":1}
I20260809 23:00:11.642277 4252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da58e6dbacfa47fa8288076312c321d6" candidate_uuid: "edf1ca3af72e4c458e15db56b5881e51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2d19f6aa18d436c943668bc4ec3a643" is_pre_election: true
I20260809 23:00:11.642441 4496 raft_consensus.cc:2781] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:00:11.642601 4252 raft_consensus.cc:2379] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate edf1ca3af72e4c458e15db56b5881e51 in current term 1: Already voted for candidate cb59e5af09b64e9490c9f7225b8116df in this term.
I20260809 23:00:11.642714 4412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da58e6dbacfa47fa8288076312c321d6" candidate_uuid: "edf1ca3af72e4c458e15db56b5881e51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb59e5af09b64e9490c9f7225b8116df" is_pre_election: true
I20260809 23:00:11.643214 4496 raft_consensus.cc:686] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 1 LEADER]: Becoming Leader. State: Replica: cb59e5af09b64e9490c9f7225b8116df, State: Running, Role: LEADER
I20260809 23:00:11.643749 4496 consensus_queue.cc:237] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } }
I20260809 23:00:11.647934 4017 leader_election.cc:302] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: edf1ca3af72e4c458e15db56b5881e51; no voters: cb59e5af09b64e9490c9f7225b8116df, e2d19f6aa18d436c943668bc4ec3a643
I20260809 23:00:11.648348 4497 raft_consensus.cc:2726] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 23:00:11.649092 3932 catalog_manager.cc:5495] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df reported cstate change: term changed from 0 to 1, leader changed from <none> to cb59e5af09b64e9490c9f7225b8116df (127.24.31.195). New cstate: current_term: 1 leader_uuid: "cb59e5af09b64e9490c9f7225b8116df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb59e5af09b64e9490c9f7225b8116df" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 36889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 } health_report { overall_health: UNKNOWN } } }
I20260809 23:00:11.724002 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.718422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.24.31.195:36889)
0809 23:00:11.718651 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.718657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.718811 (+ 154us) server_negotiation.cc:407] Connection header received
0809 23:00:11.719036 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.719040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.719121 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.719284 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:00:11.719664 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.720418 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.721807 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.722202 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.722342 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.722844 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:00:11.723335 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:00:11.723506 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.723565 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.723650 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260809 23:00:11.731998 4252 raft_consensus.cc:1261] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 1 FOLLOWER]: Refusing update from remote peer cb59e5af09b64e9490c9f7225b8116df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:00:11.732133 4090 raft_consensus.cc:1261] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 1 FOLLOWER]: Refusing update from remote peer cb59e5af09b64e9490c9f7225b8116df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:00:11.732918 4502 consensus_queue.cc:1035] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2d19f6aa18d436c943668bc4ec3a643" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 42743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:00:11.733372 4496 consensus_queue.cc:1035] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf1ca3af72e4c458e15db56b5881e51" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 45769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:00:11.745883 4507 mvcc.cc:205] Tried to move back new op lower bound from 7316752022444589056 to 7316752022096539648. Current Snapshot: MvccSnapshot[applied={T|T < 7316752022444589056}]
I20260809 23:00:11.752544 4508 mvcc.cc:205] Tried to move back new op lower bound from 7316752022444589056 to 7316752022096539648. Current Snapshot: MvccSnapshot[applied={T|T < 7316752022444589056}]
I20260809 23:00:11.768111 3932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:00:11.768390 3932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:00:11.772039 3932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f38dc860d8f146368e8f5b6e85ad6f8b: Sending DeleteTablet for 3 replicas of tablet da58e6dbacfa47fa8288076312c321d6
I20260809 23:00:11.773363 4384 tablet_service.cc:1514] Processing DeleteTablet for tablet da58e6dbacfa47fa8288076312c321d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:00:11 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:54682
I20260809 23:00:11.773460 4224 tablet_service.cc:1514] Processing DeleteTablet for tablet da58e6dbacfa47fa8288076312c321d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:00:11 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:47342
I20260809 23:00:11.774124 4068 tablet_service.cc:1514] Processing DeleteTablet for tablet da58e6dbacfa47fa8288076312c321d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:00:11 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:46470
I20260809 23:00:11.774413 4516 tablet_replica.cc:330] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: stopping tablet replica
I20260809 23:00:11.775250 4516 raft_consensus.cc:2229] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:00:11.775977 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 4000
I20260809 23:00:11.776253 4516 raft_consensus.cc:2258] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:00:11.777683 4517 tablet_replica.cc:330] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: stopping tablet replica
I20260809 23:00:11.778224 4517 raft_consensus.cc:2229] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:00:11.778903 4518 tablet_replica.cc:330] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: stopping tablet replica
I20260809 23:00:11.779419 4517 raft_consensus.cc:2258] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:00:11.779934 4518 raft_consensus.cc:2229] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:00:11.780736 4518 raft_consensus.cc:2258] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:00:11.781431 4516 ts_tablet_manager.cc:1905] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:00:11.782748 4518 ts_tablet_manager.cc:1905] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:00:11.783833 4517 ts_tablet_manager.cc:1905] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:00:11.789049 4516 ts_tablet_manager.cc:1918] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:00:11.789254 4516 log.cc:1196] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/da58e6dbacfa47fa8288076312c321d6
I20260809 23:00:11.789812 4516 ts_tablet_manager.cc:1939] T da58e6dbacfa47fa8288076312c321d6 P cb59e5af09b64e9490c9f7225b8116df: Deleting consensus metadata
I20260809 23:00:11.790164 4518 ts_tablet_manager.cc:1918] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:00:11.790396 4518 log.cc:1196] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/da58e6dbacfa47fa8288076312c321d6
I20260809 23:00:11.791013 4518 ts_tablet_manager.cc:1939] T da58e6dbacfa47fa8288076312c321d6 P edf1ca3af72e4c458e15db56b5881e51: Deleting consensus metadata
I20260809 23:00:11.791026 4517 ts_tablet_manager.cc:1918] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:00:11.791184 4517 log.cc:1196] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/da58e6dbacfa47fa8288076312c321d6
I20260809 23:00:11.791750 4517 ts_tablet_manager.cc:1939] T da58e6dbacfa47fa8288076312c321d6 P e2d19f6aa18d436c943668bc4ec3a643: Deleting consensus metadata
I20260809 23:00:11.791909 3914 catalog_manager.cc:4841] TS cb59e5af09b64e9490c9f7225b8116df (127.24.31.195:36889): tablet da58e6dbacfa47fa8288076312c321d6 (table test-table [id=66cc009f949d4853ae56eaa480e40ef6]) successfully deleted
I20260809 23:00:11.793193 3917 catalog_manager.cc:4841] TS edf1ca3af72e4c458e15db56b5881e51 (127.24.31.193:45769): tablet da58e6dbacfa47fa8288076312c321d6 (table test-table [id=66cc009f949d4853ae56eaa480e40ef6]) successfully deleted
I20260809 23:00:11.793440 3917 catalog_manager.cc:4841] TS e2d19f6aa18d436c943668bc4ec3a643 (127.24.31.194:42743): tablet da58e6dbacfa47fa8288076312c321d6 (table test-table [id=66cc009f949d4853ae56eaa480e40ef6]) successfully deleted
Aug 09 23:00:11 dist-test-slave-vxj2 krb5kdc[3882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 23:00:11.806777 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.777151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.24.31.193:45769)
0809 23:00:11.777400 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.777409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.779131 (+ 1722us) server_negotiation.cc:407] Connection header received
0809 23:00:11.779440 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.779446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.779544 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.779629 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:11.792479 (+ 12850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.793655 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.795568 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.795952 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.796043 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.800788 (+ 4745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:11.800807 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:11.800822 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:11.800865 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:11.803753 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:11.804480 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:11.804487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:11.804490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:11.804621 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:11.805077 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:11.805086 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:11.805088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:11.805514 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:11.805839 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.806141 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.806390 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 23:00:11.807425 4140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:11.902992 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 4000
I20260809 23:00:11.930132 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 4151
Aug 09 23:00:11 dist-test-slave-vxj2 krb5kdc[3882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 23:00:11.948393 4487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:11.931035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47358 (local address 127.24.31.194:42743)
0809 23:00:11.931234 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:11.931240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:11.931260 (+ 20us) server_negotiation.cc:407] Connection header received
0809 23:00:11.931423 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:11.931428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:11.931531 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:11.931650 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:11.932855 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.933925 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.935298 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:11.935963 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:11.936101 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:11.941138 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:11.941176 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:11.941198 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:11.941245 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:11.945122 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:11.946145 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:11.946155 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:11.946159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:11.946318 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:11.946710 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:11.946716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:11.946719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:11.947185 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:11.947503 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:11.947783 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 23:00:11.948036 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 23:00:11.949306 4299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:12.043013 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 4151
I20260809 23:00:12.063058 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 4313
Aug 09 23:00:12 dist-test-slave-vxj2 krb5kdc[3882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 23:00:12.082190 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:12.063941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.24.31.195:36889)
0809 23:00:12.064125 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:12.064132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:12.064209 (+ 77us) server_negotiation.cc:407] Connection header received
0809 23:00:12.064465 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:12.064469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:12.064534 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:12.064662 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:12.066177 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:12.066945 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:12.068343 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:12.068814 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:12.068961 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:12.073918 (+ 4957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:12.073939 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:12.073976 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:12.074043 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:12.078767 (+ 4724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:12.079582 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:12.079592 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:12.079595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:12.079761 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:12.080237 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:12.080242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:12.080246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:12.080790 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:12.081131 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:12.081525 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 23:00:12.081779 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 23:00:12.083052 4461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:12.225855 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 4313
I20260809 23:00:12.246254 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 3896
I20260809 23:00:12.247684 3966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:00:12.361267 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 3896
2026-08-09T23:00:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4672 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:00:12 dist-test-slave-vxj2 krb5kdc[4526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:00:12 dist-test-slave-vxj2 krb5kdc[4526](info): set up 2 sockets
Aug 09 23:00:12 dist-test-slave-vxj2 krb5kdc[4526](info): commencing operation
krb5kdc: starting...
W20260809 23:00:13.550526 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:00:13 dist-test-slave-vxj2 krb5kdc[4526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:00:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:00:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.31.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.31.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/rangeradmin_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/rangeradmin_127.24.31.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.31.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.31.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/rangerlookup_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/rangerlookup_127.24.31.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.31.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.31.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/HTTP_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/HTTP_127.24.31.212@KRBTEST.COM.keytab.
I20260809 23:00:13.921336 24703 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskHiIJT1/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/postgres -l logfile start
2026-08-09 23:00:16.523 UTC [4560] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 23:00:16.523 UTC [4560] LOG: listening on IPv4 address "127.24.31.212", port 45041
2026-08-09 23:00:16.527 UTC [4560] LOG: listening on Unix socket "/tmp/.s.PGSQL.45041"
2026-08-09 23:00:16.550 UTC [4563] LOG: database system was shut down at 2026-08-09 23:00:15 UTC
2026-08-09 23:00:16.554 UTC [4560] LOG: database system is ready to accept connections
I20260809 23:00:17.595302 24703 mini_postgres.cc:96] Postgres bound to 45041
2026-08-09 23:00:17.616 UTC [4574] FATAL: database "slave" does not exist
127.24.31.212:45041 - accepting connections
I20260809 23:00:17.617380 24703 mini_ranger.cc:161] Starting Ranger...
I20260809 23:00:17.648046 24703 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 23:00:18.162927 24703 mini_ranger.cc:87] Created ranger Postgres database
I20260809 23:00:18.163044 24703 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-admin
2026-08-09 23:00:18,458 [I] DB FLAVOR :POSTGRES
2026-08-09 23:00:18,458 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:00:18,458 [I] Checking connection..
2026-08-09 23:00:18,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 23:00:18,731 [I] Checking connection passed.
2026-08-09 23:00:18,731 [I] --------- Verifying version history table ---------
2026-08-09 23:00:18,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:00:19,002 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:00:19,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:00:19,273 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:00:19,273 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 23:00:19,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 23:00:19.538 UTC [4647] WARNING: there is no transaction in progress
2026-08-09 23:00:19,545 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 23:00:19,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:00:19,798 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 23:00:19,798 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:00:19,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 23:00:20,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 23:00:20,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vxj2.c.gcp-upstream.internal','N') ;"
2026-08-09 23:00:20,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:00:20,831 [I] Table x_portal_user does not exist in database ranger
2026-08-09 23:00:20,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:00:21,100 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 23:00:21,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:00:21,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:00:21,618 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 23:00:21,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 23:00:22.624 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.640 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.657 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.673 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.692 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.711 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.777 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.784 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.799 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.810 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.821 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.836 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.846 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.853 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.860 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:22.867 UTC [4770] WARNING: there is no transaction in progress
2026-08-09 23:00:23,541 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 23:00:23,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 23:00:23,803 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 23:00:23,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:00:24,047 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 23:00:24,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:00:24,320 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 23:00:24,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:00:24,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:00:24,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vxj2.c.gcp-upstream.internal';"
2026-08-09 23:00:25,080 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 23:00:25,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 23:00:25,325 [I] DB_PATCHES have already been applied
I20260809 23:00:25.331638 24703 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-admin:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskHiIJT1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 23:00:25.331732 24703 mini_ranger.cc:193] Using host: 127.24.31.212
I20260809 23:00:25.350639 24703 mini_ranger.cc:227] Ranger admin URL: http://127.24.31.212:42167
Aug 09, 2026 11:00:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:00:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:00:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 11:00:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 11:00:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 11:00:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 11:00:26 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 23:00:26.579322 24703 mini_ranger.cc:160] Time spent starting Ranger: real 8.962s user 0.002s sys 0.049s
Aug 09, 2026 11:00:47 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 11:00:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 11:00:47 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 11:00:47 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 11:00:48 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 23:00:48.772140 24703 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.31.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.31.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/rangerkms_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/rangerkms_127.24.31.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.31.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.31.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.31.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/HTTP_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.31.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/HTTP_127.24.31.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 23:00:48 dist-test-slave-vxj2 krb5kdc[4526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316448, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 23:00:48.986944 24703 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 23:00:49.015122 24703 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 23:00:49.920365 24703 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 23:00:49.920609 24703 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-kms
I20260809 23:00:49.920749 24703 mini_ranger_kms.cc:227] Using postgres at 127.24.31.212:45041
2026-08-09 23:00:50,404 [I] DB FLAVOR :POSTGRES
2026-08-09 23:00:50,404 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:00:50,404 [I] Checking connection
2026-08-09 23:00:50,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 23:00:50,648 [I] connection success
2026-08-09 23:00:50,648 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 23:00:50,649 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 23:00:50,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 23:00:50,883 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 23:00:50,883 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:00:50,883 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 23:00:50,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:45041/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 23:00:51,140 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 23:00:51.631012 24703 mini_ranger_kms.cc:310] Created kms service
I20260809 23:00:51.838686 24703 mini_ranger_kms.cc:326] Created kudu user
I20260809 23:00:51.886965 24703 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 23:00:51 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316448, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:00:52.310284 24703 mini_ranger_kms.cc:384] Added ranger policy
I20260809 23:00:52.312062 24703 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-kms:/tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 23:00:52.312144 24703 mini_ranger_kms.cc:240] Using host: 127.24.31.212
I20260809 23:00:52.338500 24703 mini_ranger_kms.cc:276] Ranger KMS PID: 4978
I20260809 23:00:52.338725 24703 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.31.212:58611
Aug 09, 2026 11:00:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:00:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:00:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 11:00:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 11:00:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 11:00:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 23:00:54 dist-test-slave-vxj2 krb5kdc[4526](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316454, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.31.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:00:54.942677 24703 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.956s user 0.148s sys 0.376s
I20260809 23:00:54.943033 24703 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 23:00:54.943301 24703 mini_ranger_kms.cc:401] 127.24.31.212:58611/kms/v1/keys
Aug 09 23:00:55 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316454, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.31.212@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
Aug 09 23:00:55 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316454, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.31.212@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
Aug 09 23:00:58 dist-test-slave-vxj2 krb5kdc[4526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:00:58.153101 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:40789
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:40789
--ranger_config_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:58611/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:00:58.498565 5035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:00:58.498904 5035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:00:58.499091 5035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:00:58.506441 5035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:00:58.506603 5035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:00:58.506711 5035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:00:58.506804 5035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:00:58.506893 5035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:00:58.517668 5035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.31.212:58611/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:40789
--ranger_config_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:40789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:00:58.519032 5035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:00:58.520309 5035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:00:58.533625 5041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:58.533663 5043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:00:58.533751 5045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:00:58.534379 5035 server_base.cc:1034] running on GCE node
I20260809 23:00:58.535175 5035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:00:58.536561 5035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:00:58.537778 5035 hybrid_clock.cc:648] HybridClock initialized: now 1786316458537738 us; error 52 us; skew 500 ppm
I20260809 23:00:58.538118 5035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:00:58 dist-test-slave-vxj2 krb5kdc[4526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316458, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:00:58.544723 5035 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:00:58.546307 5035 webserver.cc:460] Webserver started at http://127.24.31.254:40549/ using document root <none> and password file <none>
I20260809 23:00:58.546978 5035 fs_manager.cc:359] Metadata directory not provided
I20260809 23:00:58.547111 5035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:00:58.547427 5035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:00:58 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316458, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:00:58.604916 5035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "358249a86069437ba885d52889f5a1bd"
format_stamp: "Formatted at 2026-08-09 23:00:58 on dist-test-slave-vxj2"
server_key: "3d8e44cc0c62cd1176915f19506d9660"
server_key_iv: "495e504c1f6cb63c0376e0acfee638e5"
server_key_version: "kuduclusterkey@0"
I20260809 23:00:58.605851 5035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "358249a86069437ba885d52889f5a1bd"
format_stamp: "Formatted at 2026-08-09 23:00:58 on dist-test-slave-vxj2"
server_key: "3d8e44cc0c62cd1176915f19506d9660"
server_key_iv: "495e504c1f6cb63c0376e0acfee638e5"
server_key_version: "kuduclusterkey@0"
I20260809 23:00:58.611502 5035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:00:58.628417 5053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:00:58.629978 5035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:00:58.630173 5035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "358249a86069437ba885d52889f5a1bd"
format_stamp: "Formatted at 2026-08-09 23:00:58 on dist-test-slave-vxj2"
server_key: "3d8e44cc0c62cd1176915f19506d9660"
server_key_iv: "495e504c1f6cb63c0376e0acfee638e5"
server_key_version: "kuduclusterkey@0"
I20260809 23:00:58.630414 5035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:00:58.651202 5035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:00:58.655267 5035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:00:58.655593 5035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:00:58.677174 5035 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:40789
I20260809 23:00:58.677210 5113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:40789 every 8 connection(s)
I20260809 23:00:58.678604 5035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:00:58.682631 5115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:00:58.687912 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 5035
I20260809 23:00:58.688119 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:00:58.689060 24703 external_mini_cluster.cc:1428] Setting key 17a46ee62648e73b5cbb75337a47bc4a
I20260809 23:00:58.696622 5115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd: Bootstrap starting.
I20260809 23:00:58.702054 5115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd: Neither blocks nor log segments found. Creating new log.
I20260809 23:00:58.704006 5115 log.cc:824] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd: Log is configured to *not* fsync() on all Append() calls
I20260809 23:00:58.709439 5115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd: No bootstrap required, opened a new log
I20260809 23:00:58.719120 5115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358249a86069437ba885d52889f5a1bd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40789 } }
I20260809 23:00:58.719636 5115 raft_consensus.cc:374] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:00:58.719762 5115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 358249a86069437ba885d52889f5a1bd, State: Initialized, Role: FOLLOWER
I20260809 23:00:58.720518 5115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358249a86069437ba885d52889f5a1bd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40789 } }
I20260809 23:00:58.720837 5115 raft_consensus.cc:388] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:00:58.720990 5115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:00:58.721213 5115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [term 0 FOLLOWER]: Advancing to term 1
Aug 09 23:00:58 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:00:58.724174 5115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358249a86069437ba885d52889f5a1bd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40789 } }
I20260809 23:00:58.724754 5115 leader_election.cc:302] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 358249a86069437ba885d52889f5a1bd; no voters:
I20260809 23:00:58.727061 5115 leader_election.cc:288] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:00:58.727174 5120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:00:58.733222 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:00:58.691541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41970 (local address 127.24.31.254:40789)
0809 23:00:58.706630 (+ 15089us) server_negotiation.cc:206] Beginning negotiation
0809 23:00:58.706652 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 23:00:58.706707 (+ 55us) server_negotiation.cc:407] Connection header received
0809 23:00:58.709262 (+ 2555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:00:58.709293 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:00:58.709810 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:00:58.710309 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:00:58.712087 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:58.714245 (+ 2158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:58.716187 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:00:58.717383 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:00:58.717607 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:00:58.722828 (+ 5221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:00:58.722841 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:00:58.722862 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:00:58.722920 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:00:58.727060 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:58.728008 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:58.728017 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:58.728026 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:58.728211 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:00:58.728744 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:00:58.728748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:00:58.728752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:00:58.729530 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:00:58.731082 (+ 1552us) server_negotiation.cc:1035] Waiting for connection context
0809 23:00:58.731374 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 23:00:58.731813 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14798,"thread_start_us":14560,"threads_started":1}
I20260809 23:00:58.737767 5120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [term 1 LEADER]: Becoming Leader. State: Replica: 358249a86069437ba885d52889f5a1bd, State: Running, Role: LEADER
I20260809 23:00:58.738459 5120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358249a86069437ba885d52889f5a1bd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40789 } }
I20260809 23:00:58.739164 5115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:00:58.741506 5122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "358249a86069437ba885d52889f5a1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358249a86069437ba885d52889f5a1bd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40789 } } }
I20260809 23:00:58.741986 5122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [sys.catalog]: This master's current role is: LEADER
I20260809 23:00:58.743139 5126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:00:58.743021 5121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 358249a86069437ba885d52889f5a1bd. Latest consensus state: current_term: 1 leader_uuid: "358249a86069437ba885d52889f5a1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "358249a86069437ba885d52889f5a1bd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40789 } } }
I20260809 23:00:58.743489 5121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd [sys.catalog]: This master's current role is: LEADER
I20260809 23:00:58.765925 5126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:00:58.789834 5126 catalog_manager.cc:1347] Generated new cluster ID: 354e17554f06467f834eda04d77d2a8c
I20260809 23:00:58.789973 5126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:00:58.810715 5126 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:00:58.812749 5126 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:00:58.817203 5115 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 23:00:58.844365 5126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd: Generated new TSK 0
I20260809 23:00:58.845646 5126 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 23:00:59.648867 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:58611/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40789
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:00:59 dist-test-slave-vxj2 krb5kdc[4526](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316459, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 23:01:00.078466 5166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:00.078907 5166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:00.079101 5166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:00.091038 5166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:00.091256 5166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:00.091493 5166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:01:00.107558 5166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.31.212:58611/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:00.109450 5166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:00.111325 5166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:00.128070 5178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:00.129818 5179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:00.137880 5181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:00.138468 5166 server_base.cc:1034] running on GCE node
I20260809 23:01:00.139709 5166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:00.146966 5166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:00.148656 5166 hybrid_clock.cc:648] HybridClock initialized: now 1786316460148541 us; error 118 us; skew 500 ppm
I20260809 23:01:00.149215 5166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:00 dist-test-slave-vxj2 krb5kdc[4526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316460, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:00.154435 5166 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:01:00.160112 5166 webserver.cc:460] Webserver started at http://127.24.31.193:41025/ using document root <none> and password file <none>
I20260809 23:01:00.161274 5166 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:00.161513 5166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:01:00.161855 5166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:01:00 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316460, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:00.180001 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "c2c538f81f8441ccacb682ad661dd769"
format_stamp: "Formatted at 2026-08-09 23:01:00 on dist-test-slave-vxj2"
server_key: "c53a67464c2b7675b6db0659a75cb3d5"
server_key_iv: "7c1b5c60c88626d51cc23b712132c2aa"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:00.181403 5166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "c2c538f81f8441ccacb682ad661dd769"
format_stamp: "Formatted at 2026-08-09 23:01:00 on dist-test-slave-vxj2"
server_key: "c53a67464c2b7675b6db0659a75cb3d5"
server_key_iv: "7c1b5c60c88626d51cc23b712132c2aa"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:00.191385 5166 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.004s
I20260809 23:01:00.213909 5190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:00.236012 5166 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.002s sys 0.003s
I20260809 23:01:00.236385 5166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "c2c538f81f8441ccacb682ad661dd769"
format_stamp: "Formatted at 2026-08-09 23:01:00 on dist-test-slave-vxj2"
server_key: "c53a67464c2b7675b6db0659a75cb3d5"
server_key_iv: "7c1b5c60c88626d51cc23b712132c2aa"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:00.236879 5166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:00.291201 5166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:00.297201 5166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:00.297638 5166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:00.305224 5166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:01:00.325524 5166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:01:00.325726 5166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:00.325968 5166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:01:00.326114 5166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:01:00 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316459, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
Aug 09 23:01:00 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316459, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:00.892694 5166 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:43993
I20260809 23:01:00.892735 5333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:43993 every 8 connection(s)
I20260809 23:01:00.894349 5166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:01:00.894936 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 5166
I20260809 23:01:00.895200 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:01:00.896119 24703 external_mini_cluster.cc:1428] Setting key ef104d6c66015c5f9cf12c738d7699ff
Aug 09 23:01:00 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316460, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
I20260809 23:01:00.943344 5341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:00.901920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:40789 (local address 127.24.31.193:58361)
0809 23:01:00.913196 (+ 11276us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:00.913261 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:00.915878 (+ 2617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:00.919565 (+ 3687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:00.919574 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:00.920066 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:01:00.920668 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:00.920686 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:00.922267 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:00.922272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:00.923731 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:00.923758 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:00.926636 (+ 2878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:00.926644 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:00.926898 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:00.927591 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:01:00.927614 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:01:00.929720 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:01:00.936005 (+ 6285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:00.936017 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:00.936037 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:00.936558 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:00.938841 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:00.938847 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:00.938851 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:00.939194 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:00.940987 (+ 1793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:01:00.940995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:01:00.941516 (+ 521us) client_negotiation.cc:769] Sending connection context
0809 23:01:00.941762 (+ 246us) client_negotiation.cc:240] Negotiation successful
0809 23:01:00.942199 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10882,"thread_start_us":14278,"threads_started":1}
I20260809 23:01:00.945488 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:00.901281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:58361 (local address 127.24.31.254:40789)
0809 23:01:00.918994 (+ 17713us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:00.919001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:00.919023 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:01:00.919088 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:00.919094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:00.919194 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:00.919320 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:00.920864 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:00.922083 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:00.925414 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:00.926373 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:00.926525 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:00.929947 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:00.929952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:00.929955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:00.929989 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:00.934795 (+ 4806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:00.937480 (+ 2685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:00.937593 (+ 113us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:00.937599 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:00.937796 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:00.939356 (+ 1560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:00.939360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:00.939363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:00.940773 (+ 1410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:00.944659 (+ 3886us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:00.944800 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 23:01:00.945060 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17560,"thread_start_us":17546,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
I20260809 23:01:00.947062 5334 heartbeater.cc:347] Connected to a master server at 127.24.31.254:40789
I20260809 23:01:00.947399 5334 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:00.948287 5334 heartbeater.cc:511] Master 127.24.31.254:40789 requested a full tablet report, sending...
I20260809 23:01:00.950950 5074 ts_manager.cc:194] Registered new tserver with Master: c2c538f81f8441ccacb682ad661dd769 (127.24.31.193:43993)
I20260809 23:01:00.953177 5074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:58361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 23:01:00.998622 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:58611/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40789
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:01:01.407459 5346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:01.407754 5346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:01.407851 5346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:01.415397 5346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:01.415514 5346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:01.415627 5346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:01:01.425558 5346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.31.212:58611/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:01.426831 5346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:01.428077 5346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:01.441126 5358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:01.441248 5359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:01.441141 5361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:01.442327 5346 server_base.cc:1034] running on GCE node
I20260809 23:01:01.442925 5346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:01.443861 5346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:01.445101 5346 hybrid_clock.cc:648] HybridClock initialized: now 1786316461445056 us; error 32 us; skew 500 ppm
I20260809 23:01:01.445436 5346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:01 dist-test-slave-vxj2 krb5kdc[4526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316461, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:01.448594 5346 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:01:01.450111 5346 webserver.cc:460] Webserver started at http://127.24.31.194:45987/ using document root <none> and password file <none>
I20260809 23:01:01.450826 5346 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:01.450930 5346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:01:01.451195 5346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:01:01 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316461, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:01.465943 5346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "7b03d9c78399427a91eeb42383da6f4b"
format_stamp: "Formatted at 2026-08-09 23:01:01 on dist-test-slave-vxj2"
server_key: "9a6295d48b3b2cb3767da37c93e566c2"
server_key_iv: "a3c085fb17ca6e3deff633095b0a0294"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:01.466820 5346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "7b03d9c78399427a91eeb42383da6f4b"
format_stamp: "Formatted at 2026-08-09 23:01:01 on dist-test-slave-vxj2"
server_key: "9a6295d48b3b2cb3767da37c93e566c2"
server_key_iv: "a3c085fb17ca6e3deff633095b0a0294"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:01.472273 5346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 23:01:01.487135 5369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:01.488691 5346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:01:01.488891 5346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "7b03d9c78399427a91eeb42383da6f4b"
format_stamp: "Formatted at 2026-08-09 23:01:01 on dist-test-slave-vxj2"
server_key: "9a6295d48b3b2cb3767da37c93e566c2"
server_key_iv: "a3c085fb17ca6e3deff633095b0a0294"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:01.489131 5346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:01.509431 5346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:01.513301 5346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:01.513633 5346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:01.514632 5346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:01:01.516062 5346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:01:01.516168 5346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:01.516322 5346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:01:01.516431 5346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:01.551465 5346 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:43253
I20260809 23:01:01.551512 5501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:43253 every 8 connection(s)
I20260809 23:01:01.552834 5346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:01:01.562763 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 5346
I20260809 23:01:01.562999 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:01:01.563458 24703 external_mini_cluster.cc:1428] Setting key b048bffea11106995c578956b9cf4ce8
Aug 09 23:01:01 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316461, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:01:01.573606 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:01.555664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:39989 (local address 127.24.31.254:40789)
0809 23:01:01.556157 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:01.556162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:01.557216 (+ 1054us) server_negotiation.cc:407] Connection header received
0809 23:01:01.559153 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:01.559159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:01.559252 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:01.559372 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:01.560842 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:01.561689 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:01.563466 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:01.563913 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:01.564049 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:01.567594 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:01.567599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:01.567602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:01.567634 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:01.570411 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:01.571168 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:01.571174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:01.571178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:01.571306 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:01.571736 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:01.571742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:01.571745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:01.572034 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:01.572197 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:01.572931 (+ 734us) server_negotiation.cc:299] Negotiation successful
0809 23:01:01.573215 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":262,"threads_started":1}
I20260809 23:01:01.574661 5505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:01.556283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:40789 (local address 127.24.31.194:39989)
0809 23:01:01.557032 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:01.557071 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:01.558853 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:01.559624 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:01.559632 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:01.560093 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:01:01.560671 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:01.560687 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:01.561851 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:01.561855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:01.563311 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:01.563331 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:01.564138 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:01.564146 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:01.564388 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:01.565058 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:01:01.565080 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:01:01.567368 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:01:01.570573 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:01.570582 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:01.570598 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:01.571049 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:01.571416 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:01.571421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:01.571425 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:01.571626 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:01.572174 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:01:01.572183 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:01:01.572552 (+ 369us) client_negotiation.cc:769] Sending connection context
0809 23:01:01.572834 (+ 282us) client_negotiation.cc:240] Negotiation successful
0809 23:01:01.573413 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":293,"threads_started":1}
I20260809 23:01:01.576188 5502 heartbeater.cc:347] Connected to a master server at 127.24.31.254:40789
I20260809 23:01:01.576514 5502 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:01.577276 5502 heartbeater.cc:511] Master 127.24.31.254:40789 requested a full tablet report, sending...
I20260809 23:01:01.578778 5074 ts_manager.cc:194] Registered new tserver with Master: 7b03d9c78399427a91eeb42383da6f4b (127.24.31.194:43253)
I20260809 23:01:01.579571 5074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:39989
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 23:01:01.624099 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:58611/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40789
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:01:01.957209 5334 heartbeater.cc:503] Master 127.24.31.254:40789 was elected leader, sending a full tablet report...
W20260809 23:01:01.975646 5509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:01.975944 5509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:01.976081 5509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:01.983654 5509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:01.983801 5509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:01.983994 5509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:01:01.994226 5509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.31.212:58611/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:40789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:01.995548 5509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:01.996938 5509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:02.010689 5521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:02.010622 5518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:02.010715 5519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:02.011396 5509 server_base.cc:1034] running on GCE node
I20260809 23:01:02.012022 5509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:02.012974 5509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:02.014201 5509 hybrid_clock.cc:648] HybridClock initialized: now 1786316462014153 us; error 53 us; skew 500 ppm
I20260809 23:01:02.014535 5509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:02 dist-test-slave-vxj2 krb5kdc[4526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316462, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:02.017697 5509 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:01:02.019265 5509 webserver.cc:460] Webserver started at http://127.24.31.195:37599/ using document root <none> and password file <none>
I20260809 23:01:02.019945 5509 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:02.020092 5509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:01:02.020411 5509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:01:02 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316462, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:02.034818 5509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "dcb630e4557041b683a1b2874494375d"
format_stamp: "Formatted at 2026-08-09 23:01:02 on dist-test-slave-vxj2"
server_key: "4ed6e9a925e41a8132babc0b9a823f0c"
server_key_iv: "30c0b722581e24f1329544039896dd1c"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:02.035750 5509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "dcb630e4557041b683a1b2874494375d"
format_stamp: "Formatted at 2026-08-09 23:01:02 on dist-test-slave-vxj2"
server_key: "4ed6e9a925e41a8132babc0b9a823f0c"
server_key_iv: "30c0b722581e24f1329544039896dd1c"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:02.041544 5509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 23:01:02.056376 5531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:02.057960 5509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 23:01:02.058149 5509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "dcb630e4557041b683a1b2874494375d"
format_stamp: "Formatted at 2026-08-09 23:01:02 on dist-test-slave-vxj2"
server_key: "4ed6e9a925e41a8132babc0b9a823f0c"
server_key_iv: "30c0b722581e24f1329544039896dd1c"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:02.058399 5509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:02.090121 5509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:02.094147 5509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:02.094476 5509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:02.095471 5509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:01:02.096870 5509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:01:02.096978 5509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:02.097131 5509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:01:02.097220 5509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 23:01:02.136107 5509 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:38725
I20260809 23:01:02.136149 5670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:38725 every 8 connection(s)
I20260809 23:01:02.137485 5509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:01:02.141641 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 5509
I20260809 23:01:02.141839 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:01:02.142359 24703 external_mini_cluster.cc:1428] Setting key 64fcc3830fce30ab18909621b0a81526
Aug 09 23:01:02 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316462, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:01:02.159222 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.140247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:43847 (local address 127.24.31.254:40789)
0809 23:01:02.140777 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.140783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.141728 (+ 945us) server_negotiation.cc:407] Connection header received
0809 23:01:02.144111 (+ 2383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.144117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.144262 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.144437 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:02.146395 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.147238 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.148830 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.149389 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.149497 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.152888 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:02.152894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:02.152897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:02.152931 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:02.155835 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.156612 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.156618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.156621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.156822 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.157348 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.157354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.157362 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.157744 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:02.157913 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.158608 (+ 695us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.158842 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":287,"threads_started":1}
I20260809 23:01:02.160368 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.140766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:40789 (local address 127.24.31.195:43847)
0809 23:01:02.141572 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:02.141616 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:02.143728 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:02.144680 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:02.144694 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:02.145349 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:01:02.146228 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.146251 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.147402 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.147405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.148689 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.148701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.149590 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.149595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.149802 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.150516 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:01:02.150550 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:01:02.152638 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:01:02.155978 (+ 3340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:02.155984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:02.155999 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:02.156471 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:02.156947 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:02.156960 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:02.156967 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:02.157232 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:02.157883 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:01:02.157891 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:01:02.158233 (+ 342us) client_negotiation.cc:769] Sending connection context
0809 23:01:02.158511 (+ 278us) client_negotiation.cc:240] Negotiation successful
0809 23:01:02.159129 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":367,"threads_started":1}
I20260809 23:01:02.161906 5671 heartbeater.cc:347] Connected to a master server at 127.24.31.254:40789
I20260809 23:01:02.162258 5671 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:02.163151 5671 heartbeater.cc:511] Master 127.24.31.254:40789 requested a full tablet report, sending...
I20260809 23:01:02.164681 5074 ts_manager.cc:194] Registered new tserver with Master: dcb630e4557041b683a1b2874494375d (127.24.31.195:38725)
I20260809 23:01:02.165580 5074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:43847
I20260809 23:01:02.171243 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:01:02.188977 5675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.176106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41982 (local address 127.24.31.254:40789)
0809 23:01:02.176271 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.176275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.176862 (+ 587us) server_negotiation.cc:407] Connection header received
0809 23:01:02.176963 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.176967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.177045 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.177128 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:02.178608 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.179327 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.180672 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.181127 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.181315 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.183713 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:02.183724 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:02.183727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:02.183759 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:02.186238 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.186918 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.186924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.186927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.187062 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.187438 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.187445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.187448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.187713 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:02.187977 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.188397 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.188621 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:01:02.193096 5074 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:01:02.200634 5074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:01:02.241725 5690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.214056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:43993 (local address 127.0.0.1:48852)
0809 23:01:02.215056 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:02.215107 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:02.215334 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:02.233371 (+ 18037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:02.233377 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:02.233417 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:02.234036 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.234047 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.236233 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.236243 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.239042 (+ 2799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.239059 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.240523 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.240530 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.240774 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.240828 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 23:01:02.240994 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 23:01:02.241230 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":102,"thread_start_us":655,"threads_started":1}
I20260809 23:01:02.242401 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.214999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43708 (local address 127.24.31.195:38725)
0809 23:01:02.232424 (+ 17425us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.232433 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.232474 (+ 41us) server_negotiation.cc:407] Connection header received
0809 23:01:02.232569 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.232577 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.232812 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.233026 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:02.234710 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.235876 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.239050 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.240436 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.240548 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.240812 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.241839 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.242032 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17193,"thread_start_us":16984,"threads_started":1}
I20260809 23:01:02.242401 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.216002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.24.31.194:43253)
0809 23:01:02.231302 (+ 15300us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.231315 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.232142 (+ 827us) server_negotiation.cc:407] Connection header received
0809 23:01:02.232262 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.232274 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.232624 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.232840 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:02.234992 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.236390 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.238872 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.240393 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.240549 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.241220 (+ 671us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.241839 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.242025 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15063,"thread_start_us":14901,"threads_started":1}
I20260809 23:01:02.242879 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.214379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48852 (local address 127.24.31.193:43993)
0809 23:01:02.232724 (+ 18345us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.232736 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.232766 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:01:02.232861 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.232867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.233080 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.233646 (+ 566us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:02.234239 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.236075 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.239195 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.240290 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.240434 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.240727 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.242278 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.242470 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18159,"thread_start_us":17981,"threads_started":1}
I20260809 23:01:02.243008 5692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.215652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:43253 (local address 127.0.0.1:39860)
0809 23:01:02.231828 (+ 16176us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:02.231853 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:02.232020 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:02.233199 (+ 1179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:02.233208 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:02.233244 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:02.234638 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.234655 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.236566 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.236571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.238688 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.238702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.241037 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.241042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.241384 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.241424 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:01:02.241497 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:01:02.241601 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16012,"mutex_wait_us":284,"thread_start_us":13597,"threads_started":1}
I20260809 23:01:02.243077 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.214569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:38725 (local address 127.0.0.1:43708)
0809 23:01:02.228879 (+ 14310us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:02.228911 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:02.229121 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:02.233226 (+ 4105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:02.233233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:02.233280 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:02.234497 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.234511 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.236009 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.236015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.238880 (+ 2865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.238896 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.241008 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.241021 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.241264 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.241300 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 23:01:02.241390 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 23:01:02.241532 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14108,"mutex_wait_us":9,"thread_start_us":16953,"threads_started":1}
I20260809 23:01:02.246688 5423 tablet_service.cc:1467] Processing CreateTablet for tablet bce080c24d0d42fe9da89ac3840b2000 (DEFAULT_TABLE table=test-table [id=26188c1631da43119bd112b35161f115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:01:02.246762 5589 tablet_service.cc:1467] Processing CreateTablet for tablet bce080c24d0d42fe9da89ac3840b2000 (DEFAULT_TABLE table=test-table [id=26188c1631da43119bd112b35161f115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:01:02.246768 5252 tablet_service.cc:1467] Processing CreateTablet for tablet bce080c24d0d42fe9da89ac3840b2000 (DEFAULT_TABLE table=test-table [id=26188c1631da43119bd112b35161f115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:01:02.248003 5423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bce080c24d0d42fe9da89ac3840b2000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:01:02.248209 5252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bce080c24d0d42fe9da89ac3840b2000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:01:02.248209 5589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bce080c24d0d42fe9da89ac3840b2000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:01:02.259590 5697 tablet_bootstrap.cc:492] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: Bootstrap starting.
I20260809 23:01:02.263393 5697 tablet_bootstrap.cc:654] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: Neither blocks nor log segments found. Creating new log.
I20260809 23:01:02.264437 5697 log.cc:824] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:01:02.267290 5697 tablet_bootstrap.cc:492] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: No bootstrap required, opened a new log
I20260809 23:01:02.267552 5697 ts_tablet_manager.cc:1397] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260809 23:01:02.269959 5700 tablet_bootstrap.cc:492] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: Bootstrap starting.
I20260809 23:01:02.270267 5699 tablet_bootstrap.cc:492] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: Bootstrap starting.
I20260809 23:01:02.273703 5700 tablet_bootstrap.cc:654] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: Neither blocks nor log segments found. Creating new log.
I20260809 23:01:02.273921 5699 tablet_bootstrap.cc:654] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: Neither blocks nor log segments found. Creating new log.
I20260809 23:01:02.274958 5700 log.cc:824] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: Log is configured to *not* fsync() on all Append() calls
I20260809 23:01:02.275158 5699 log.cc:824] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:01:02.277849 5700 tablet_bootstrap.cc:492] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: No bootstrap required, opened a new log
I20260809 23:01:02.278139 5700 ts_tablet_manager.cc:1397] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260809 23:01:02.277782 5697 raft_consensus.cc:348] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } }
I20260809 23:01:02.278419 5697 raft_consensus.cc:374] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:01:02.278555 5697 raft_consensus.cc:729] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b03d9c78399427a91eeb42383da6f4b, State: Initialized, Role: FOLLOWER
I20260809 23:01:02.279383 5697 consensus_queue.cc:260] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } }
I20260809 23:01:02.280692 5502 heartbeater.cc:503] Master 127.24.31.254:40789 was elected leader, sending a full tablet report...
I20260809 23:01:02.281781 5697 ts_tablet_manager.cc:1428] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260809 23:01:02.284659 5699 tablet_bootstrap.cc:492] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: No bootstrap required, opened a new log
I20260809 23:01:02.284981 5699 ts_tablet_manager.cc:1397] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.009s
I20260809 23:01:02.289386 5700 raft_consensus.cc:348] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } }
I20260809 23:01:02.289772 5700 raft_consensus.cc:374] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:01:02.289893 5700 raft_consensus.cc:729] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2c538f81f8441ccacb682ad661dd769, State: Initialized, Role: FOLLOWER
I20260809 23:01:02.290465 5700 consensus_queue.cc:260] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } }
I20260809 23:01:02.292201 5700 ts_tablet_manager.cc:1428] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260809 23:01:02.292857 5699 raft_consensus.cc:348] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } }
I20260809 23:01:02.293375 5699 raft_consensus.cc:374] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:01:02.293491 5699 raft_consensus.cc:729] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb630e4557041b683a1b2874494375d, State: Initialized, Role: FOLLOWER
I20260809 23:01:02.294032 5699 consensus_queue.cc:260] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } }
I20260809 23:01:02.295279 5671 heartbeater.cc:503] Master 127.24.31.254:40789 was elected leader, sending a full tablet report...
I20260809 23:01:02.296130 5699 ts_tablet_manager.cc:1428] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
W20260809 23:01:02.305100 5503 tablet.cc:2369] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:01:02.389287 5672 tablet.cc:2369] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:01:02.401041 5337 tablet.cc:2369] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:01:02.408674 5706 raft_consensus.cc:482] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:01:02.408974 5706 raft_consensus.cc:504] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } }
I20260809 23:01:02.410715 5706 leader_election.cc:288] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2c538f81f8441ccacb682ad661dd769 (127.24.31.193:43993), 7b03d9c78399427a91eeb42383da6f4b (127.24.31.194:43253)
I20260809 23:01:02.417389 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.411143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:60473 (local address 127.24.31.193:43993)
0809 23:01:02.411370 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.411378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.411402 (+ 24us) server_negotiation.cc:407] Connection header received
0809 23:01:02.411488 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.411493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.411574 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.411679 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:02.412563 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.413572 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.415417 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.416546 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.416665 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.416718 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.417064 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.417197 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:01:02.417511 5676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.410993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:43993 (local address 127.24.31.195:60473)
0809 23:01:02.411153 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:02.411175 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:02.411302 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:02.411967 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:02.411971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:02.412035 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:02.412409 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.412423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.413712 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.413717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.415260 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.415272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.416689 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.416695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.416893 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.416938 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 23:01:02.417043 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 23:01:02.417209 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"spinlock_wait_cycles":5376}
I20260809 23:01:02.418514 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.411628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:47407 (local address 127.24.31.194:43253)
0809 23:01:02.411898 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.411905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.412141 (+ 236us) server_negotiation.cc:407] Connection header received
0809 23:01:02.412312 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.412317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.412414 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.412518 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:02.413216 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.414141 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.416312 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.417452 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.417607 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.417681 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.418122 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.418249 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 23:01:02.418599 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.411385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:43253 (local address 127.24.31.195:47407)
0809 23:01:02.412044 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:02.412059 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:02.412180 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:02.412550 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:02.412554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:02.412579 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:02.412999 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.413011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.414305 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.414309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.416135 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:02.416152 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.417772 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.417778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:02.417968 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.417991 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 23:01:02.418095 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 23:01:02.418220 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"spinlock_wait_cycles":20608,"thread_start_us":361,"threads_started":1}
I20260809 23:01:02.418576 5279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bce080c24d0d42fe9da89ac3840b2000" candidate_uuid: "dcb630e4557041b683a1b2874494375d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c538f81f8441ccacb682ad661dd769" is_pre_election: true
I20260809 23:01:02.419251 5279 raft_consensus.cc:2454] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcb630e4557041b683a1b2874494375d in term 0.
I20260809 23:01:02.419611 5451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bce080c24d0d42fe9da89ac3840b2000" candidate_uuid: "dcb630e4557041b683a1b2874494375d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b03d9c78399427a91eeb42383da6f4b" is_pre_election: true
I20260809 23:01:02.419926 5536 leader_election.cc:302] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2c538f81f8441ccacb682ad661dd769, dcb630e4557041b683a1b2874494375d; no voters:
I20260809 23:01:02.420076 5451 raft_consensus.cc:2454] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcb630e4557041b683a1b2874494375d in term 0.
I20260809 23:01:02.420311 5706 raft_consensus.cc:2781] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:01:02.420492 5706 raft_consensus.cc:482] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:01:02.420627 5706 raft_consensus.cc:3037] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:01:02.422900 5706 raft_consensus.cc:504] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } }
I20260809 23:01:02.423581 5706 leader_election.cc:288] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [CANDIDATE]: Term 1 election: Requested vote from peers c2c538f81f8441ccacb682ad661dd769 (127.24.31.193:43993), 7b03d9c78399427a91eeb42383da6f4b (127.24.31.194:43253)
I20260809 23:01:02.424156 5451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bce080c24d0d42fe9da89ac3840b2000" candidate_uuid: "dcb630e4557041b683a1b2874494375d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b03d9c78399427a91eeb42383da6f4b"
I20260809 23:01:02.424150 5279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bce080c24d0d42fe9da89ac3840b2000" candidate_uuid: "dcb630e4557041b683a1b2874494375d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2c538f81f8441ccacb682ad661dd769"
I20260809 23:01:02.424360 5451 raft_consensus.cc:3037] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:01:02.424394 5279 raft_consensus.cc:3037] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:01:02.426646 5451 raft_consensus.cc:2454] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcb630e4557041b683a1b2874494375d in term 1.
I20260809 23:01:02.427006 5536 leader_election.cc:302] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b03d9c78399427a91eeb42383da6f4b, dcb630e4557041b683a1b2874494375d; no voters:
I20260809 23:01:02.427022 5279 raft_consensus.cc:2454] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcb630e4557041b683a1b2874494375d in term 1.
I20260809 23:01:02.427315 5706 raft_consensus.cc:2781] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:01:02.427798 5706 raft_consensus.cc:686] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 1 LEADER]: Becoming Leader. State: Replica: dcb630e4557041b683a1b2874494375d, State: Running, Role: LEADER
I20260809 23:01:02.428244 5706 consensus_queue.cc:237] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } }
I20260809 23:01:02.432653 5073 catalog_manager.cc:5495] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d reported cstate change: term changed from 0 to 1, leader changed from <none> to dcb630e4557041b683a1b2874494375d (127.24.31.195). New cstate: current_term: 1 leader_uuid: "dcb630e4557041b683a1b2874494375d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcb630e4557041b683a1b2874494375d" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38725 } health_report { overall_health: HEALTHY } } }
I20260809 23:01:02.483893 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.478118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.24.31.195:38725)
0809 23:01:02.478331 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.478336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.478354 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:01:02.478448 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.478451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.478528 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.478656 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:01:02.479098 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.479849 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.481519 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.482006 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.482102 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.482658 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:01:02.483264 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:01:02.483381 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.483499 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.483564 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 23:01:02.492491 5451 raft_consensus.cc:1261] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [term 1 FOLLOWER]: Refusing update from remote peer dcb630e4557041b683a1b2874494375d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:01:02.492537 5279 raft_consensus.cc:1261] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [term 1 FOLLOWER]: Refusing update from remote peer dcb630e4557041b683a1b2874494375d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:01:02.493528 5709 consensus_queue.cc:1035] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2c538f81f8441ccacb682ad661dd769" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:01:02.493901 5706 consensus_queue.cc:1035] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b03d9c78399427a91eeb42383da6f4b" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:01:02.506850 5713 mvcc.cc:205] Tried to move back new op lower bound from 7316752230359035904 to 7316752230109519872. Current Snapshot: MvccSnapshot[applied={T|T < 7316752230359035904}]
I20260809 23:01:02.522363 5074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:01:02.522696 5074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:01:02.526458 5074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 358249a86069437ba885d52889f5a1bd: Sending DeleteTablet for 3 replicas of tablet bce080c24d0d42fe9da89ac3840b2000
I20260809 23:01:02.527815 5589 tablet_service.cc:1514] Processing DeleteTablet for tablet bce080c24d0d42fe9da89ac3840b2000 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:01:02 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:43708
I20260809 23:01:02.527829 5252 tablet_service.cc:1514] Processing DeleteTablet for tablet bce080c24d0d42fe9da89ac3840b2000 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:01:02 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:48852
I20260809 23:01:02.528139 5423 tablet_service.cc:1514] Processing DeleteTablet for tablet bce080c24d0d42fe9da89ac3840b2000 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:01:02 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:39860
I20260809 23:01:02.529791 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 5166
I20260809 23:01:02.540666 5725 tablet_replica.cc:330] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: stopping tablet replica
I20260809 23:01:02.541216 5725 raft_consensus.cc:2229] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:01:02.541882 5725 raft_consensus.cc:2258] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:01:02.542807 5724 tablet_replica.cc:330] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: stopping tablet replica
I20260809 23:01:02.543242 5724 raft_consensus.cc:2229] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:01:02.543800 5724 raft_consensus.cc:2258] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:01:02.544641 5723 tablet_replica.cc:330] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: stopping tablet replica
I20260809 23:01:02.544961 5725 ts_tablet_manager.cc:1905] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:01:02.545105 5723 raft_consensus.cc:2229] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:01:02.546082 5723 raft_consensus.cc:2258] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:01:02.547026 5724 ts_tablet_manager.cc:1905] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:01:02.549100 5723 ts_tablet_manager.cc:1905] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:01:02.556881 5723 ts_tablet_manager.cc:1918] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:01:02.557129 5723 log.cc:1196] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/bce080c24d0d42fe9da89ac3840b2000
I20260809 23:01:02.557686 5723 ts_tablet_manager.cc:1939] T bce080c24d0d42fe9da89ac3840b2000 P dcb630e4557041b683a1b2874494375d: Deleting consensus metadata
I20260809 23:01:02.559763 5724 ts_tablet_manager.cc:1918] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:01:02.559756 5058 catalog_manager.cc:4841] TS dcb630e4557041b683a1b2874494375d (127.24.31.195:38725): tablet bce080c24d0d42fe9da89ac3840b2000 (table test-table [id=26188c1631da43119bd112b35161f115]) successfully deleted
I20260809 23:01:02.559953 5724 log.cc:1196] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/bce080c24d0d42fe9da89ac3840b2000
I20260809 23:01:02.560511 5724 ts_tablet_manager.cc:1939] T bce080c24d0d42fe9da89ac3840b2000 P c2c538f81f8441ccacb682ad661dd769: Deleting consensus metadata
I20260809 23:01:02.562477 5056 catalog_manager.cc:4841] TS c2c538f81f8441ccacb682ad661dd769 (127.24.31.193:43993): tablet bce080c24d0d42fe9da89ac3840b2000 (table test-table [id=26188c1631da43119bd112b35161f115]) successfully deleted
I20260809 23:01:02.562978 5725 ts_tablet_manager.cc:1918] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:01:02.563171 5725 log.cc:1196] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/bce080c24d0d42fe9da89ac3840b2000
I20260809 23:01:02.563728 5725 ts_tablet_manager.cc:1939] T bce080c24d0d42fe9da89ac3840b2000 P 7b03d9c78399427a91eeb42383da6f4b: Deleting consensus metadata
I20260809 23:01:02.565603 5056 catalog_manager.cc:4841] TS 7b03d9c78399427a91eeb42383da6f4b (127.24.31.194:43253): tablet bce080c24d0d42fe9da89ac3840b2000 (table test-table [id=26188c1631da43119bd112b35161f115]) successfully deleted
Aug 09 23:01:02 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 23:01:02.573760 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.531616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48856 (local address 127.24.31.193:43993)
0809 23:01:02.531840 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.531850 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.542241 (+ 10391us) server_negotiation.cc:407] Connection header received
0809 23:01:02.544906 (+ 2665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.544913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.545005 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.545095 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:02.547469 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.548658 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.556167 (+ 7509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.556809 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.556917 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.567315 (+ 10398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:02.567328 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:02.567343 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:02.567386 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:02.570310 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.571057 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.571065 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.571067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.571200 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.571605 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.571610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.571617 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.571987 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:02.572312 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.572622 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.573047 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":71}
I20260809 23:01:02.574471 5329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:02.665771 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 5166
I20260809 23:01:02.695163 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 5346
Aug 09 23:01:02 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 23:01:02.719178 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.696294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39864 (local address 127.24.31.194:43253)
0809 23:01:02.696985 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.696992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.697015 (+ 23us) server_negotiation.cc:407] Connection header received
0809 23:01:02.697208 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.697226 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.697316 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.697440 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:02.699528 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.700990 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.702982 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.703622 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.703756 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.710945 (+ 7189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:02.710968 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:02.710992 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:02.711067 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:02.715236 (+ 4169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.716281 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.716292 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.716298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.716469 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.716970 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.716976 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.716979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.717513 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:02.717808 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.718477 (+ 669us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.718752 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484}
I20260809 23:01:02.719931 5496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:02.874332 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 5346
I20260809 23:01:02.901543 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 5509
Aug 09 23:01:02 dist-test-slave-vxj2 krb5kdc[4526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 23:01:02.921201 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:02.902453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43738 (local address 127.24.31.195:38725)
0809 23:01:02.902671 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:02.902678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:02.902705 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:01:02.902800 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:02.902812 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:02.902907 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:02.903047 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:02.904415 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.905821 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.907236 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:02.908031 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:02.908202 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:02.912986 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:02.913009 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:02.913032 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:02.913093 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:02.917763 (+ 4670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.918625 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.918636 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.918640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.918843 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:02.919346 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:02.919352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:02.919355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:02.919879 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:02.920238 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:02.920494 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 23:01:02.920733 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 23:01:02.922076 5666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:03.050554 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 5509
I20260809 23:01:03.073192 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 5035
I20260809 23:01:03.074492 5109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:03.188539 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 5035
I20260809 23:01:03.218683 24703 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 23:01:03.565073 24703 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 23:01:03.565443 24703 mini_ranger.cc:66] Stopping Ranger...
I20260809 23:01:03.910802 24703 mini_ranger.cc:68] Stopped Ranger
2026-08-09 23:01:03.911 UTC [4560] LOG: received smart shutdown request
2026-08-09 23:01:03.915 UTC [4560] LOG: background worker "logical replication launcher" (PID 4571) exited with exit code 1
2026-08-09 23:01:03.915 UTC [4565] LOG: shutting down
2026-08-09 23:01:04.091 UTC [4560] LOG: database system is shut down
2026-08-09T23:01:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51853 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:01:04 dist-test-slave-vxj2 krb5kdc[5741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:01:04 dist-test-slave-vxj2 krb5kdc[5741](info): set up 2 sockets
Aug 09 23:01:04 dist-test-slave-vxj2 krb5kdc[5741](info): commencing operation
krb5kdc: starting...
W20260809 23:01:05.408799 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:01:05 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:01:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:01:05Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.31.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.31.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/rangeradmin_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/rangeradmin_127.24.31.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.31.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.31.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/rangerlookup_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/rangerlookup_127.24.31.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.31.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.31.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/HTTP_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/HTTP_127.24.31.212@KRBTEST.COM.keytab.
I20260809 23:01:05.777530 24703 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskHiIJT1/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/postgres -l logfile start
2026-08-09 23:01:08.239 UTC [5777] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 23:01:08.239 UTC [5777] LOG: listening on IPv4 address "127.24.31.212", port 39471
2026-08-09 23:01:08.243 UTC [5777] LOG: listening on Unix socket "/tmp/.s.PGSQL.39471"
2026-08-09 23:01:08.266 UTC [5780] LOG: database system was shut down at 2026-08-09 23:01:06 UTC
2026-08-09 23:01:08.270 UTC [5777] LOG: database system is ready to accept connections
I20260809 23:01:09.318128 24703 mini_postgres.cc:96] Postgres bound to 39471
2026-08-09 23:01:09.338 UTC [5789] FATAL: database "slave" does not exist
127.24.31.212:39471 - accepting connections
I20260809 23:01:09.339414 24703 mini_ranger.cc:161] Starting Ranger...
I20260809 23:01:09.367274 24703 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 23:01:09.842587 24703 mini_ranger.cc:87] Created ranger Postgres database
I20260809 23:01:09.842724 24703 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-admin
2026-08-09 23:01:10,154 [I] DB FLAVOR :POSTGRES
2026-08-09 23:01:10,155 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:01:10,155 [I] Checking connection..
2026-08-09 23:01:10,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 23:01:10,425 [I] Checking connection passed.
2026-08-09 23:01:10,425 [I] --------- Verifying version history table ---------
2026-08-09 23:01:10,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:01:10,662 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:01:10,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:01:10,926 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:01:10,927 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 23:01:10,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 23:01:11.168 UTC [5863] WARNING: there is no transaction in progress
2026-08-09 23:01:11,176 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 23:01:11,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:01:11,442 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 23:01:11,442 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:01:11,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 23:01:11,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 23:01:11,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vxj2.c.gcp-upstream.internal','N') ;"
2026-08-09 23:01:12,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:01:12,498 [I] Table x_portal_user does not exist in database ranger
2026-08-09 23:01:12,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:01:12,775 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 23:01:12,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:01:13,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:01:13,298 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 23:01:13,299 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 23:01:14.255 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.272 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.289 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.304 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.321 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.338 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.396 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.403 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.417 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.428 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.438 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.452 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.461 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.468 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.475 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:14.481 UTC [5986] WARNING: there is no transaction in progress
2026-08-09 23:01:15,115 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 23:01:15,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 23:01:15,368 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 23:01:15,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:01:15,608 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 23:01:15,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:01:15,848 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 23:01:15,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:01:16,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:01:16,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vxj2.c.gcp-upstream.internal';"
2026-08-09 23:01:16,601 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 23:01:16,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 23:01:16,841 [I] DB_PATCHES have already been applied
I20260809 23:01:16.847865 24703 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-admin:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskHiIJT1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 23:01:16.847960 24703 mini_ranger.cc:193] Using host: 127.24.31.212
I20260809 23:01:16.866513 24703 mini_ranger.cc:227] Ranger admin URL: http://127.24.31.212:59957
Aug 09, 2026 11:01:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:01:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:01:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 11:01:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 11:01:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 11:01:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 11:01:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 23:01:18.098798 24703 mini_ranger.cc:160] Time spent starting Ranger: real 8.759s user 0.001s sys 0.046s
Aug 09, 2026 11:01:39 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 11:01:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 11:01:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 11:01:39 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 11:01:40 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 23:01:40.628490 24703 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.31.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.31.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/rangerkms_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.31.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/rangerkms_127.24.31.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.31.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.31.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.31.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/HTTP_127.24.31.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.31.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/HTTP_127.24.31.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 23:01:40 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316500, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 23:01:40.825692 24703 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 23:01:40.850898 24703 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 23:01:41.758261 24703 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 23:01:41.758358 24703 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-kms
I20260809 23:01:41.758389 24703 mini_ranger_kms.cc:227] Using postgres at 127.24.31.212:39471
2026-08-09 23:01:42,129 [I] DB FLAVOR :POSTGRES
2026-08-09 23:01:42,129 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:01:42,129 [I] Checking connection
2026-08-09 23:01:42,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 23:01:42,359 [I] connection success
2026-08-09 23:01:42,359 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 23:01:42,359 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 23:01:42,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 23:01:42,592 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 23:01:42,593 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:01:42,593 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 23:01:42,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.31.212:39471/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 23:01:42,851 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 23:01:43.340754 24703 mini_ranger_kms.cc:310] Created kms service
I20260809 23:01:43.534452 24703 mini_ranger_kms.cc:326] Created kudu user
I20260809 23:01:43.580238 24703 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 23:01:43 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316500, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:43.974146 24703 mini_ranger_kms.cc:384] Added ranger policy
I20260809 23:01:43.974473 24703 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-kms:/tmp/dist-test-taskHiIJT1/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskHiIJT1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 23:01:43.974555 24703 mini_ranger_kms.cc:240] Using host: 127.24.31.212
I20260809 23:01:43.993336 24703 mini_ranger_kms.cc:276] Ranger KMS PID: 6200
I20260809 23:01:43.993468 24703 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.31.212:33289
Aug 09, 2026 11:01:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:01:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:01:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 11:01:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 11:01:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 11:01:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 23:01:46 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316506, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.31.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:46.604708 24703 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.779s user 0.038s sys 0.361s
I20260809 23:01:46.604991 24703 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 23:01:46.605212 24703 mini_ranger_kms.cc:401] 127.24.31.212:33289/kms/v1/keys
Aug 09 23:01:46 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316506, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.31.212@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
Aug 09 23:01:47 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316506, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.31.212@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
Aug 09 23:01:49 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:01:49.563405 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39487
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39487
--ranger_config_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:33289/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:01:49.916973 6256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:49.917258 6256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:49.917433 6256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:49.924454 6256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:01:49.924575 6256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:49.924679 6256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:49.924790 6256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:01:49.924868 6256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:01:49.934773 6256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.31.212:33289/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39487
--ranger_config_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:49.935998 6256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:49.937386 6256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:49.949791 6263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:49.949842 6264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:49.949877 6266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:49.950539 6256 server_base.cc:1034] running on GCE node
I20260809 23:01:49.951296 6256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:49.952613 6256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:49.953853 6256 hybrid_clock.cc:648] HybridClock initialized: now 1786316509953813 us; error 49 us; skew 500 ppm
I20260809 23:01:49.954186 6256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:49 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316509, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:49.966706 6256 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:01:49.968240 6256 webserver.cc:460] Webserver started at http://127.24.31.254:39537/ using document root <none> and password file <none>
I20260809 23:01:49.968895 6256 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:49.969027 6256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:01:49.969394 6256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:01:49 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316509, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:50.026430 6256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "d31743ec2f6e49ab80de61bd6e2c7b76"
format_stamp: "Formatted at 2026-08-09 23:01:50 on dist-test-slave-vxj2"
server_key: "9d75ceeeedd157f66f9ee5c2394b46d9"
server_key_iv: "e52f552c28d277212bb1289025947d16"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:50.027392 6256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "d31743ec2f6e49ab80de61bd6e2c7b76"
format_stamp: "Formatted at 2026-08-09 23:01:50 on dist-test-slave-vxj2"
server_key: "9d75ceeeedd157f66f9ee5c2394b46d9"
server_key_iv: "e52f552c28d277212bb1289025947d16"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:50.032891 6256 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 23:01:50.052529 6273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:50.054071 6256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 23:01:50.054284 6256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "d31743ec2f6e49ab80de61bd6e2c7b76"
format_stamp: "Formatted at 2026-08-09 23:01:50 on dist-test-slave-vxj2"
server_key: "9d75ceeeedd157f66f9ee5c2394b46d9"
server_key_iv: "e52f552c28d277212bb1289025947d16"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:50.054525 6256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:50.096786 6256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:50.100530 6256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:50.100857 6256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:50.120685 6256 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:39487
I20260809 23:01:50.120710 6337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:39487 every 8 connection(s)
I20260809 23:01:50.122133 6256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:01:50.123379 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 6256
I20260809 23:01:50.123615 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:01:50.124503 24703 external_mini_cluster.cc:1428] Setting key b75fe4c4c7fb7ddc45b4cfe813616cf3
I20260809 23:01:50.126909 6338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 23:01:50 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:01:50.140096 6338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Bootstrap starting.
I20260809 23:01:50.144464 6338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Neither blocks nor log segments found. Creating new log.
I20260809 23:01:50.145758 6338 log.cc:824] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Log is configured to *not* fsync() on all Append() calls
I20260809 23:01:50.148540 6341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:50.126580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50586 (local address 127.24.31.254:39487)
0809 23:01:50.127405 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:50.127416 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:50.127453 (+ 37us) server_negotiation.cc:407] Connection header received
0809 23:01:50.129050 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:50.129070 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:50.129516 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:50.129821 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:50.131658 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:50.132973 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:50.134713 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:50.135355 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:50.135475 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:50.140170 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:50.140202 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:50.140219 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:50.140269 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:50.144161 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:50.144893 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:50.144903 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:50.144916 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:50.145098 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:50.145486 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:50.145494 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:50.145500 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:50.146214 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:50.146581 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:50.146948 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 23:01:50.147289 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":429,"threads_started":1}
I20260809 23:01:50.149209 6338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: No bootstrap required, opened a new log
I20260809 23:01:50.156577 6338 raft_consensus.cc:348] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } }
I20260809 23:01:50.156910 6338 raft_consensus.cc:374] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:01:50.157114 6338 raft_consensus.cc:729] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31743ec2f6e49ab80de61bd6e2c7b76, State: Initialized, Role: FOLLOWER
I20260809 23:01:50.157809 6338 consensus_queue.cc:260] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } }
I20260809 23:01:50.158083 6338 raft_consensus.cc:388] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:01:50.158232 6338 raft_consensus.cc:482] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:01:50.158428 6338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:01:50.160699 6338 raft_consensus.cc:504] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } }
I20260809 23:01:50.161343 6338 leader_election.cc:302] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d31743ec2f6e49ab80de61bd6e2c7b76; no voters:
I20260809 23:01:50.161903 6338 leader_election.cc:288] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:01:50.162009 6343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:01:50.162799 6343 raft_consensus.cc:686] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 1 LEADER]: Becoming Leader. State: Replica: d31743ec2f6e49ab80de61bd6e2c7b76, State: Running, Role: LEADER
I20260809 23:01:50.163331 6343 consensus_queue.cc:237] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } }
I20260809 23:01:50.164047 6338 sys_catalog.cc:564] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:01:50.166134 6345 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d31743ec2f6e49ab80de61bd6e2c7b76. Latest consensus state: current_term: 1 leader_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } } }
I20260809 23:01:50.166617 6345 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: This master's current role is: LEADER
I20260809 23:01:50.167685 6344 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } } }
I20260809 23:01:50.168048 6344 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: This master's current role is: LEADER
I20260809 23:01:50.170584 6349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:01:50.175846 6349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:01:50.176455 6338 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 23:01:50.193238 6349 catalog_manager.cc:1347] Generated new cluster ID: 14058f1f12f34ff8b9aa89bab6e0a898
I20260809 23:01:50.193543 6349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:01:50.223232 6349 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:01:50.224653 6349 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:01:50.233289 6349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Generated new TSK 0
I20260809 23:01:50.234349 6349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 23:01:50.987030 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:33289/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39487
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:01:51 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 23:01:51.419277 6390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:51.419600 6390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:51.419799 6390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:51.427534 6390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:51.427675 6390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:51.427861 6390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:01:51.439679 6390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.31.212:33289/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:51.441131 6390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:51.442484 6390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:51.471158 6406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:51.477566 6405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:51.479203 6408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:51.485466 6390 server_base.cc:1034] running on GCE node
I20260809 23:01:51.486357 6390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:51.489446 6390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:51.490782 6390 hybrid_clock.cc:648] HybridClock initialized: now 1786316511490695 us; error 56 us; skew 500 ppm
I20260809 23:01:51.491190 6390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:51 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:51.495818 6390 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:01:51.498047 6390 webserver.cc:460] Webserver started at http://127.24.31.193:32901/ using document root <none> and password file <none>
I20260809 23:01:51.498960 6390 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:51.499095 6390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:01:51.499428 6390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:01:51 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:51.529515 6390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "c777c600cf4d4999a538da92bc751d9b"
format_stamp: "Formatted at 2026-08-09 23:01:51 on dist-test-slave-vxj2"
server_key: "7e7150dca89bf7f1d4829a11052252cf"
server_key_iv: "f4acc68d717166c56378382996e845a4"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:51.531550 6390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "c777c600cf4d4999a538da92bc751d9b"
format_stamp: "Formatted at 2026-08-09 23:01:51 on dist-test-slave-vxj2"
server_key: "7e7150dca89bf7f1d4829a11052252cf"
server_key_iv: "f4acc68d717166c56378382996e845a4"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:51.541111 6390 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.002s
I20260809 23:01:51.560425 6415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:51.562434 6390 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 23:01:51.562656 6390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "c777c600cf4d4999a538da92bc751d9b"
format_stamp: "Formatted at 2026-08-09 23:01:51 on dist-test-slave-vxj2"
server_key: "7e7150dca89bf7f1d4829a11052252cf"
server_key_iv: "f4acc68d717166c56378382996e845a4"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:51.562964 6390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:51.600005 6390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:51.605456 6390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:51.605804 6390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:51.607164 6390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:01:51.608999 6390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:01:51.609170 6390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:51.609380 6390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:01:51.609448 6390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:51.701591 6390 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:39985
I20260809 23:01:51.701623 6549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:39985 every 8 connection(s)
I20260809 23:01:51.703365 6390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:01:51.707949 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 6390
I20260809 23:01:51.708199 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:01:51.709256 24703 external_mini_cluster.cc:1428] Setting key 545b7af682b1dddbfea8b03b2f0878e5
Aug 09 23:01:51 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
Aug 09 23:01:51 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
I20260809 23:01:51.752597 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:51.706435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:52421 (local address 127.24.31.254:39487)
0809 23:01:51.713480 (+ 7045us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:51.713487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:51.727740 (+ 14253us) server_negotiation.cc:407] Connection header received
0809 23:01:51.731544 (+ 3804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:51.731552 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:51.731664 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:51.731778 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:51.734252 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:51.735555 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:51.738194 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:51.738909 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:51.739018 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:51.744106 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:51.744114 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:51.744119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:51.744175 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:51.748338 (+ 4163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:51.749492 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:51.749499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:51.749503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:51.749668 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:51.750265 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:51.750271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:51.750275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:51.750672 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:51.750835 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:51.751881 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0809 23:01:51.752120 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6914,"thread_start_us":10701,"threads_started":1}
I20260809 23:01:51.755009 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:51.707087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39487 (local address 127.24.31.193:52421)
0809 23:01:51.727488 (+ 20401us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:51.727563 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:51.730952 (+ 3389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:51.732414 (+ 1462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:51.732428 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:51.733174 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:01:51.734019 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:51.734047 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:51.735805 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:51.735811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:51.738002 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:51.738019 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:51.739414 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:51.739422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:51.739743 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:51.740790 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:01:51.740824 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:01:51.743779 (+ 2955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:01:51.748498 (+ 4719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:51.748509 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:51.748527 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:51.749328 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:51.749796 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:51.749801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:51.749805 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:51.750101 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:51.750942 (+ 841us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:01:51.750951 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:01:51.751393 (+ 442us) client_negotiation.cc:769] Sending connection context
0809 23:01:51.752946 (+ 1553us) client_negotiation.cc:240] Negotiation successful
0809 23:01:51.753548 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20046,"thread_start_us":19789,"threads_started":1}
I20260809 23:01:51.757942 6550 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39487
I20260809 23:01:51.758286 6550 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:51.759218 6550 heartbeater.cc:511] Master 127.24.31.254:39487 requested a full tablet report, sending...
I20260809 23:01:51.766882 6297 ts_manager.cc:194] Registered new tserver with Master: c777c600cf4d4999a538da92bc751d9b (127.24.31.193:39985)
I20260809 23:01:51.768554 6297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:52421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 23:01:51.795533 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:33289/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39487
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:01:52 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316511, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
W20260809 23:01:52.267982 6557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:52.268692 6557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:52.268874 6557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:52.280673 6557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:52.280881 6557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:52.281112 6557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:01:52.296765 6557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.31.212:33289/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:52.298718 6557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:52.300702 6557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:52.340516 6566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:52.341322 6565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:52.344636 6557 server_base.cc:1034] running on GCE node
W20260809 23:01:52.344945 6569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:52.345799 6557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:52.352350 6557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:52.353681 6557 hybrid_clock.cc:648] HybridClock initialized: now 1786316512353570 us; error 104 us; skew 500 ppm
I20260809 23:01:52.354174 6557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:52 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316512, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:52.359035 6557 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:01:52.372176 6557 webserver.cc:460] Webserver started at http://127.24.31.194:34279/ using document root <none> and password file <none>
I20260809 23:01:52.373368 6557 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:52.373524 6557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:01:52.373886 6557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:01:52 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316512, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:52.401247 6557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "f81f5a4f27a74a58ad8f51bbf266a26c"
format_stamp: "Formatted at 2026-08-09 23:01:52 on dist-test-slave-vxj2"
server_key: "2c0fd668e3693c699c766cb019c0bb20"
server_key_iv: "4b0ad74dd12b7d3326e2e06ed8fa5a19"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:52.402510 6557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "f81f5a4f27a74a58ad8f51bbf266a26c"
format_stamp: "Formatted at 2026-08-09 23:01:52 on dist-test-slave-vxj2"
server_key: "2c0fd668e3693c699c766cb019c0bb20"
server_key_iv: "4b0ad74dd12b7d3326e2e06ed8fa5a19"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:52.410646 6557 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260809 23:01:52.436372 6582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:52.438117 6557 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 23:01:52.438351 6557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "f81f5a4f27a74a58ad8f51bbf266a26c"
format_stamp: "Formatted at 2026-08-09 23:01:52 on dist-test-slave-vxj2"
server_key: "2c0fd668e3693c699c766cb019c0bb20"
server_key_iv: "4b0ad74dd12b7d3326e2e06ed8fa5a19"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:52.438618 6557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:52.483908 6557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:52.488612 6557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:52.489005 6557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:52.490257 6557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:01:52.491910 6557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:01:52.492026 6557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:52.492199 6557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:01:52.492290 6557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:52.542368 6557 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:35599
I20260809 23:01:52.542335 6717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:35599 every 8 connection(s)
I20260809 23:01:52.543910 6557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:01:52.545876 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 6557
I20260809 23:01:52.546093 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:01:52.546845 24703 external_mini_cluster.cc:1428] Setting key 0625fc42c9431643b65c469a33ea910a
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Aug 09 23:01:52 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316512, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:01:52.604611 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:52.550711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:43797 (local address 127.24.31.254:39487)
0809 23:01:52.573574 (+ 22863us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:52.573582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:52.573629 (+ 47us) server_negotiation.cc:407] Connection header received
0809 23:01:52.582774 (+ 9145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:52.582782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:52.582902 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:52.582993 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:52.585434 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:52.586595 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:52.588933 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:52.589572 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:52.589663 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:52.596865 (+ 7202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:52.596872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:52.596877 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:52.596921 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:52.600671 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:52.601714 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:52.601720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:52.601724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:52.601889 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:52.602485 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:52.602491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:52.602494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:52.602850 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:52.602981 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:52.603956 (+ 975us) server_negotiation.cc:299] Negotiation successful
0809 23:01:52.604193 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22700,"thread_start_us":22911,"threads_started":1}
I20260809 23:01:52.607070 6722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:52.551657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39487 (local address 127.24.31.194:43797)
0809 23:01:52.571247 (+ 19590us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:52.571323 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:52.582335 (+ 11012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:52.583636 (+ 1301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:52.583649 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:52.584326 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:01:52.585188 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:52.585216 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:52.586809 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:52.586814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:52.588770 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:52.588784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:52.590015 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:52.590022 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:52.590308 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:52.591095 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:01:52.591125 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:01:52.596584 (+ 5459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:01:52.600894 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:52.600904 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:52.600922 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:52.601565 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:52.602044 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:52.602049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:52.602052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:52.602349 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:52.603084 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:01:52.603092 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:01:52.603535 (+ 443us) client_negotiation.cc:769] Sending connection context
0809 23:01:52.605050 (+ 1515us) client_negotiation.cc:240] Negotiation successful
0809 23:01:52.605647 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19221,"thread_start_us":18952,"threads_started":1}
I20260809 23:01:52.608929 6718 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39487
I20260809 23:01:52.609279 6718 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:52.610121 6718 heartbeater.cc:511] Master 127.24.31.254:39487 requested a full tablet report, sending...
I20260809 23:01:52.611924 6297 ts_manager.cc:194] Registered new tserver with Master: f81f5a4f27a74a58ad8f51bbf266a26c (127.24.31.194:35599)
I20260809 23:01:52.613026 6297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:43797
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 23:01:52.636471 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:33289/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39487
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:01:52.772457 6550 heartbeater.cc:503] Master 127.24.31.254:39487 was elected leader, sending a full tablet report...
W20260809 23:01:53.048210 6725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:53.048480 6725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:53.048635 6725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:53.055902 6725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:53.056090 6725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:53.056283 6725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:01:53.066655 6725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.31.212:33289/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:53.068048 6725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:53.069572 6725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:53.084009 6732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:53.084055 6731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:53.084122 6734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:53.085340 6725 server_base.cc:1034] running on GCE node
I20260809 23:01:53.085984 6725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:53.087029 6725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:53.088285 6725 hybrid_clock.cc:648] HybridClock initialized: now 1786316513088231 us; error 43 us; skew 500 ppm
I20260809 23:01:53.088652 6725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:53 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316513, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:53.091956 6725 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:01:53.093703 6725 webserver.cc:460] Webserver started at http://127.24.31.195:37161/ using document root <none> and password file <none>
I20260809 23:01:53.094391 6725 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:53.094523 6725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:01:53.094846 6725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:01:53 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316513, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:53.113056 6725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "774f5d5c291d46f5b69f051d2d2faaf3"
format_stamp: "Formatted at 2026-08-09 23:01:53 on dist-test-slave-vxj2"
server_key: "9828b240c3de6703d9c1341cb550c056"
server_key_iv: "09b8fe7749a38851f30965c9f8c1dd20"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:53.114162 6725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "774f5d5c291d46f5b69f051d2d2faaf3"
format_stamp: "Formatted at 2026-08-09 23:01:53 on dist-test-slave-vxj2"
server_key: "9828b240c3de6703d9c1341cb550c056"
server_key_iv: "09b8fe7749a38851f30965c9f8c1dd20"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:53.120939 6725 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 23:01:53.142038 6743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:53.143507 6725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:01:53.143689 6725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "774f5d5c291d46f5b69f051d2d2faaf3"
format_stamp: "Formatted at 2026-08-09 23:01:53 on dist-test-slave-vxj2"
server_key: "9828b240c3de6703d9c1341cb550c056"
server_key_iv: "09b8fe7749a38851f30965c9f8c1dd20"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:53.143931 6725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:53.159487 6725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:53.163097 6725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:53.163414 6725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:53.164393 6725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:01:53.165910 6725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:01:53.166014 6725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:53.166168 6725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:01:53.166242 6725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:53.199270 6725 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:46189
I20260809 23:01:53.199303 6876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:46189 every 8 connection(s)
I20260809 23:01:53.200685 6725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:01:53.210501 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 6725
I20260809 23:01:53.210698 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:01:53.211154 24703 external_mini_cluster.cc:1428] Setting key b202986ae9f44d29f3eb1e369f7aea7c
Aug 09 23:01:53 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316513, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:01:53.219926 6879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:53.203205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:42187 (local address 127.24.31.254:39487)
0809 23:01:53.203742 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:53.203747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:53.204494 (+ 747us) server_negotiation.cc:407] Connection header received
0809 23:01:53.206343 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:53.206348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:53.206475 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:53.206602 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:53.207989 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.208739 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.210350 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.210868 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.211004 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:53.214823 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:53.214828 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:53.214831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:53.214862 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:53.217339 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:53.218048 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:53.218052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:53.218054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:53.218189 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:53.218556 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:53.218559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:53.218561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:53.218803 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:53.218909 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:53.219510 (+ 601us) server_negotiation.cc:299] Negotiation successful
0809 23:01:53.219666 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":287,"threads_started":1}
I20260809 23:01:53.220899 6880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:53.203613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39487 (local address 127.24.31.195:42187)
0809 23:01:53.204303 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:53.204345 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:53.206078 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:53.206831 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:53.206838 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:53.207252 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:01:53.207819 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:53.207836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.208900 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.208905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:53.210194 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:53.210207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.211102 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.211109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:53.211380 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:53.212047 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:01:53.212069 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:01:53.214632 (+ 2563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:01:53.217460 (+ 2828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:53.217466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:53.217476 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:53.217947 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:53.218278 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:53.218281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:53.218283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:53.218460 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:53.218908 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:01:53.218914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:01:53.219193 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 23:01:53.219414 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 23:01:53.219763 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":307,"threads_started":1}
I20260809 23:01:53.222255 6877 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39487
I20260809 23:01:53.222546 6877 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:53.223210 6877 heartbeater.cc:511] Master 127.24.31.254:39487 requested a full tablet report, sending...
I20260809 23:01:53.224618 6297 ts_manager.cc:194] Registered new tserver with Master: 774f5d5c291d46f5b69f051d2d2faaf3 (127.24.31.195:46189)
I20260809 23:01:53.225700 6297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:42187
I20260809 23:01:53.226718 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:01:53.228430 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 6390
Aug 09 23:01:53 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 23:01:53.244791 6883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:53.229477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.24.31.193:39985)
0809 23:01:53.230137 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:53.230143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:53.230166 (+ 23us) server_negotiation.cc:407] Connection header received
0809 23:01:53.230218 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:53.230224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:53.230405 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:53.230517 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:53.231908 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.232892 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.234512 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.235000 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.235091 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:53.239383 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:53.239394 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:53.239405 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:53.239440 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:53.242110 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:53.242777 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:53.242783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:53.242785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:53.242921 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:53.243262 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:53.243265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:53.243267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:53.243641 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:53.243859 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:53.244161 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 23:01:53.244424 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":342,"threads_started":1}
I20260809 23:01:53.246017 6544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:53.328392 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 6390
I20260809 23:01:53.347538 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 6557
Aug 09 23:01:53 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 23:01:53.363876 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:53.348580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33182 (local address 127.24.31.194:35599)
0809 23:01:53.349110 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:53.349117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:53.349143 (+ 26us) server_negotiation.cc:407] Connection header received
0809 23:01:53.349261 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:53.349268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:53.349488 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:53.349614 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:53.350847 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.352032 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.353439 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.353940 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.354039 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:53.358323 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:53.358335 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:53.358349 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:53.358381 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:53.361331 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:53.362022 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:53.362029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:53.362031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:53.362166 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:53.362477 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:53.362480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:53.362482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:53.362815 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:53.363048 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:53.363284 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 23:01:53.363475 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":260,"threads_started":1}
I20260809 23:01:53.365329 6713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:53.450958 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 6557
I20260809 23:01:53.469761 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 6725
Aug 09 23:01:53 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 23:01:53.491951 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:53.475274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.24.31.195:46189)
0809 23:01:53.475834 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:53.475841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:53.475871 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:01:53.475960 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:53.475966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:53.476174 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:53.476369 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:53.478066 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.479113 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.481183 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:53.481870 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:53.482018 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:53.486659 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:53.486671 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:53.486685 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:53.486723 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:53.489562 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:53.490229 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:53.490235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:53.490238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:53.490371 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:53.490672 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:53.490675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:53.490677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:53.490999 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:53.491242 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:53.491477 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 23:01:53.491671 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":257,"threads_started":1}
I20260809 23:01:53.493265 6870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:53.576581 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 6725
I20260809 23:01:53.594730 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 6256
I20260809 23:01:53.596113 6333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:53.705294 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 6256
I20260809 23:01:53.734404 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39487
--webserver_interface=127.24.31.254
--webserver_port=39537
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39487
--ranger_config_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:33289/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.31.212:33289/kms with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:01:54.060521 6891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:54.060864 6891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:54.061043 6891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:54.068360 6891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:01:54.068518 6891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:54.068634 6891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:54.068740 6891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:01:54.068835 6891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:01:54.078958 6891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.31.212:33289/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39487
--ranger_config_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=39537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:54.080288 6891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:54.081912 6891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:54.094450 6898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:54.094542 6897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:54.094933 6900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:54.095358 6891 server_base.cc:1034] running on GCE node
I20260809 23:01:54.096158 6891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:54.097637 6891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:54.098886 6891 hybrid_clock.cc:648] HybridClock initialized: now 1786316514098835 us; error 47 us; skew 500 ppm
I20260809 23:01:54.099231 6891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:54 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316514, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:54.102357 6891 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:01:54.103821 6891 webserver.cc:460] Webserver started at http://127.24.31.254:39537/ using document root <none> and password file <none>
I20260809 23:01:54.104493 6891 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:54.104652 6891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:01:54 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316514, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:54.153074 6891 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 23:01:54.156188 6909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:54.157603 6891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 23:01:54.157805 6891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "d31743ec2f6e49ab80de61bd6e2c7b76"
format_stamp: "Formatted at 2026-08-09 23:01:50 on dist-test-slave-vxj2"
server_key: "9d75ceeeedd157f66f9ee5c2394b46d9"
server_key_iv: "e52f552c28d277212bb1289025947d16"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:54.158859 6891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:54.170629 6891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:54.174726 6891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:54.175014 6891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:54.193915 6891 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:39487
I20260809 23:01:54.193951 6973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:39487 every 8 connection(s)
I20260809 23:01:54.195479 6891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:01:54.199574 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 6891
I20260809 23:01:54.200454 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:33289/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:39985
--local_ip_for_outbound_sockets=127.24.31.193
--tserver_master_addrs=127.24.31.254:39487
--webserver_port=32901
--webserver_interface=127.24.31.193
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.31.212:33289/kms with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:01:54.202508 6974 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:01:54.205766 6974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Bootstrap starting.
I20260809 23:01:54.224740 6974 log.cc:824] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Log is configured to *not* fsync() on all Append() calls
I20260809 23:01:54.234467 6974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:01:54.235103 6974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Bootstrap complete.
I20260809 23:01:54.244670 6974 raft_consensus.cc:348] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } }
I20260809 23:01:54.245621 6974 raft_consensus.cc:729] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d31743ec2f6e49ab80de61bd6e2c7b76, State: Initialized, Role: FOLLOWER
I20260809 23:01:54.246227 6974 consensus_queue.cc:260] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } }
I20260809 23:01:54.246486 6974 raft_consensus.cc:388] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:01:54.246652 6974 raft_consensus.cc:482] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:01:54.246836 6974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:01:54.250720 6974 raft_consensus.cc:504] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } }
I20260809 23:01:54.251199 6974 leader_election.cc:302] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d31743ec2f6e49ab80de61bd6e2c7b76; no voters:
I20260809 23:01:54.251845 6974 leader_election.cc:288] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:01:54.252039 6980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:01:54.253531 6980 raft_consensus.cc:686] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [term 2 LEADER]: Becoming Leader. State: Replica: d31743ec2f6e49ab80de61bd6e2c7b76, State: Running, Role: LEADER
I20260809 23:01:54.254271 6974 sys_catalog.cc:564] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:01:54.254168 6980 consensus_queue.cc:237] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } }
I20260809 23:01:54.256916 6981 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } } }
I20260809 23:01:54.257416 6981 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: This master's current role is: LEADER
I20260809 23:01:54.257477 6982 sys_catalog.cc:455] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d31743ec2f6e49ab80de61bd6e2c7b76. Latest consensus state: current_term: 2 leader_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d31743ec2f6e49ab80de61bd6e2c7b76" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39487 } } }
I20260809 23:01:54.257798 6982 sys_catalog.cc:458] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76 [sys.catalog]: This master's current role is: LEADER
I20260809 23:01:54.261857 6986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:01:54.267905 6986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:01:54.269532 6986 catalog_manager.cc:1259] Loaded cluster ID: 14058f1f12f34ff8b9aa89bab6e0a898
I20260809 23:01:54.269686 6986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:01:54.273511 6986 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:01:54.274983 6986 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Loaded TSK: 0
I20260809 23:01:54.276682 6986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:01:54.282878 6974 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 23:01:54.638891 6976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:54.639358 6976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:54.639634 6976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:54.651605 6976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:54.651835 6976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:54.652092 6976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:01:54.667918 6976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.31.212:33289/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:39985
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=32901
--tserver_master_addrs=127.24.31.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:54.669847 6976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:54.671656 6976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:54.703799 7020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:54.705513 7021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:54.722982 7024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:54.723680 6976 server_base.cc:1034] running on GCE node
I20260809 23:01:54.724618 6976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:54.726096 6976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:54.727401 6976 hybrid_clock.cc:648] HybridClock initialized: now 1786316514727328 us; error 73 us; skew 500 ppm
I20260809 23:01:54.727856 6976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:54 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316514, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:54.732681 6976 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:01:54.747690 6976 webserver.cc:460] Webserver started at http://127.24.31.193:32901/ using document root <none> and password file <none>
I20260809 23:01:54.748837 6976 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:54.749095 6976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:01:54 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316514, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:54.821712 6976 fs_manager.cc:709] Time spent opening directory manager: real 0.051s user 0.017s sys 0.032s
I20260809 23:01:54.832739 7034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:54.837090 6976 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.008s
I20260809 23:01:54.837319 6976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "c777c600cf4d4999a538da92bc751d9b"
format_stamp: "Formatted at 2026-08-09 23:01:51 on dist-test-slave-vxj2"
server_key: "7e7150dca89bf7f1d4829a11052252cf"
server_key_iv: "f4acc68d717166c56378382996e845a4"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:54.838840 6976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:54.898139 6976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:54.902992 6976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:54.903291 6976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:54.906170 6976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:01:54.907776 6976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:01:54.907940 6976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:54.908144 6976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:01:54.908300 6976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:54.996718 6976 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:39985
I20260809 23:01:54.996752 7174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:39985 every 8 connection(s)
I20260809 23:01:54.998718 6976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:01:55.005968 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 6976
I20260809 23:01:55.007076 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:33289/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:35599
--local_ip_for_outbound_sockets=127.24.31.194
--tserver_master_addrs=127.24.31.254:39487
--webserver_port=34279
--webserver_interface=127.24.31.194
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.31.212:33289/kms with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:01:55 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316514, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:01:55.067044 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:55.001904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39487 (local address 127.24.31.193:46633)
0809 23:01:55.025651 (+ 23747us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:55.025727 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:55.028156 (+ 2429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:55.030881 (+ 2725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:55.030896 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:55.031592 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:01:55.032426 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:55.032452 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:55.035040 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:55.035046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:55.037021 (+ 1975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:55.037037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:55.038619 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:55.038628 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:55.038908 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:55.042905 (+ 3997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:01:55.042937 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:01:55.050841 (+ 7904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:01:55.061686 (+ 10845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:55.061696 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:55.061711 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:55.062384 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:55.062942 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:55.062947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:55.062951 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:55.063171 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:55.064287 (+ 1116us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:01:55.064295 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:01:55.064697 (+ 402us) client_negotiation.cc:769] Sending connection context
0809 23:01:55.065012 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 23:01:55.065629 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23371,"thread_start_us":23077,"threads_started":1}
I20260809 23:01:55.070142 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:55.001941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:46633 (local address 127.24.31.254:39487)
0809 23:01:55.025704 (+ 23763us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:55.025728 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:55.027887 (+ 2159us) server_negotiation.cc:407] Connection header received
0809 23:01:55.028967 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:55.028989 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:55.029556 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:55.030062 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:55.032675 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:55.034844 (+ 2169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:55.037220 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:55.038312 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:55.038538 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:55.053664 (+ 15126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:55.053684 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:55.053714 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:55.053786 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:55.061449 (+ 7663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:55.062576 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:55.062585 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:55.062593 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:55.062797 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:55.063319 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:55.063324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:55.063330 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:55.064007 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:55.064194 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:55.067767 (+ 3573us) server_negotiation.cc:299] Negotiation successful
0809 23:01:55.068158 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23451,"thread_start_us":23667,"threads_started":1}
I20260809 23:01:55.073809 7175 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39487
I20260809 23:01:55.074133 7175 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:55.075011 7175 heartbeater.cc:511] Master 127.24.31.254:39487 requested a full tablet report, sending...
W20260809 23:01:55.078372 7175 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:39487 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 23:01:55.078613 7175 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:55.078878 7175 heartbeater.cc:511] Master 127.24.31.254:39487 requested a full tablet report, sending...
I20260809 23:01:55.080284 7175 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:55.080724 7175 heartbeater.cc:511] Master 127.24.31.254:39487 requested a full tablet report, sending...
W20260809 23:01:55.082253 7175 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:01:55.423959 7180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:55.424324 7180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:55.424506 7180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:55.432513 7180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:55.432667 7180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:55.432858 7180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:01:55.443516 7180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.31.212:33289/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:35599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=34279
--tserver_master_addrs=127.24.31.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:55.444953 7180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:55.446306 7180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:55.464071 7199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:55.464933 7202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:55.470219 7204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:55.470674 7180 server_base.cc:1034] running on GCE node
I20260809 23:01:55.471364 7180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:55.472378 7180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:55.473619 7180 hybrid_clock.cc:648] HybridClock initialized: now 1786316515473553 us; error 54 us; skew 500 ppm
I20260809 23:01:55.473956 7180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:55 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:55.477448 7180 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:01:55.479255 7180 webserver.cc:460] Webserver started at http://127.24.31.194:34279/ using document root <none> and password file <none>
I20260809 23:01:55.480211 7180 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:55.480372 7180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:01:55 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:55.520220 7180 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.011s sys 0.001s
I20260809 23:01:55.524022 7213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:55.525861 7180 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 23:01:55.526070 7180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "f81f5a4f27a74a58ad8f51bbf266a26c"
format_stamp: "Formatted at 2026-08-09 23:01:52 on dist-test-slave-vxj2"
server_key: "2c0fd668e3693c699c766cb019c0bb20"
server_key_iv: "4b0ad74dd12b7d3326e2e06ed8fa5a19"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:55.527572 7180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:55.568862 7180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:55.574548 7180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:55.574882 7180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:55.576130 7180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:01:55.578047 7180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:01:55.578163 7180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:55.578279 7180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:01:55.578370 7180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:01:55 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786316515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:55.630678 7180 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:35599
I20260809 23:01:55.630724 7355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:35599 every 8 connection(s)
I20260809 23:01:55.632674 7180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:01:55.641978 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 7180
I20260809 23:01:55.642822 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.31.212:33289/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:46189
--local_ip_for_outbound_sockets=127.24.31.195
--tserver_master_addrs=127.24.31.254:39487
--webserver_port=37161
--webserver_interface=127.24.31.195
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.31.212:33289/kms with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:01:55 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:01:55.686513 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:55.637182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:45413 (local address 127.24.31.254:39487)
0809 23:01:55.652986 (+ 15804us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:55.652997 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:55.657228 (+ 4231us) server_negotiation.cc:407] Connection header received
0809 23:01:55.658100 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:55.658108 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:55.658202 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:55.658286 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:55.661037 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:55.662375 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:55.665140 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:55.665619 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:55.665694 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:55.677510 (+ 11816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:55.677519 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:55.677526 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:55.677589 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:55.681489 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:55.682493 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:55.682499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:55.682503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:55.682647 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:55.683159 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:55.683164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:55.683168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:55.683535 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:55.683649 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:55.685852 (+ 2203us) server_negotiation.cc:299] Negotiation successful
0809 23:01:55.686104 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15571,"thread_start_us":15827,"threads_started":1}
I20260809 23:01:55.689069 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:55.637949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39487 (local address 127.24.31.194:45413)
0809 23:01:55.653795 (+ 15846us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:55.653886 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:55.656998 (+ 3112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:55.659123 (+ 2125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:55.659136 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:55.659834 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:01:55.660794 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:55.660821 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:55.662613 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:55.662618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:55.664971 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:55.664987 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:55.665999 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:55.666003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:55.666193 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:55.673721 (+ 7528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:01:55.673765 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:01:55.677137 (+ 3372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:01:55.681681 (+ 4544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:55.681692 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:55.681711 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:55.682340 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:55.682792 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:55.682797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:55.682801 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:55.683033 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:55.683767 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:01:55.683776 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:01:55.685399 (+ 1623us) client_negotiation.cc:769] Sending connection context
0809 23:01:55.687071 (+ 1672us) client_negotiation.cc:240] Negotiation successful
0809 23:01:55.687669 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15495,"thread_start_us":15230,"threads_started":1}
I20260809 23:01:55.690949 7356 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39487
I20260809 23:01:55.691259 7356 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:55.692094 7356 heartbeater.cc:511] Master 127.24.31.254:39487 requested a full tablet report, sending...
I20260809 23:01:55.694128 6929 ts_manager.cc:194] Registered new tserver with Master: f81f5a4f27a74a58ad8f51bbf266a26c (127.24.31.194:35599)
I20260809 23:01:55.696254 6929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:45413
W20260809 23:01:56.066509 7360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:01:56.066836 7360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:01:56.067029 7360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:01:56.075569 7360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:01:56.075770 7360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:01:56.076035 7360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:01:56.082419 7175 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:56.082890 7175 heartbeater.cc:511] Master 127.24.31.254:39487 requested a full tablet report, sending...
I20260809 23:01:56.084444 6929 ts_manager.cc:194] Registered new tserver with Master: c777c600cf4d4999a538da92bc751d9b (127.24.31.193:39985)
I20260809 23:01:56.085599 6929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:46633
I20260809 23:01:56.088099 7360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.31.212:33289/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:46189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=37161
--tserver_master_addrs=127.24.31.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:01:56.089869 7360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:01:56.091651 7360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:01:56.125397 7367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:56.127631 7368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:01:56.129585 7371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:01:56.129803 7360 server_base.cc:1034] running on GCE node
I20260809 23:01:56.130743 7360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:01:56.132073 7360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:01:56.133354 7360 hybrid_clock.cc:648] HybridClock initialized: now 1786316516133277 us; error 53 us; skew 500 ppm
I20260809 23:01:56.133828 7360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:01:56 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
Aug 09 23:01:56 dist-test-slave-vxj2 krb5kdc[5741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:01:56.138090 7360 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:01:56.142810 7360 webserver.cc:460] Webserver started at http://127.24.31.195:37161/ using document root <none> and password file <none>
I20260809 23:01:56.143791 7360 fs_manager.cc:359] Metadata directory not provided
I20260809 23:01:56.144022 7360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:01:56 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
I20260809 23:01:56.192513 7360 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.004s
I20260809 23:01:56.209836 7380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:56.211874 7360 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.006s sys 0.010s
I20260809 23:01:56.212112 7360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "774f5d5c291d46f5b69f051d2d2faaf3"
format_stamp: "Formatted at 2026-08-09 23:01:53 on dist-test-slave-vxj2"
server_key: "9828b240c3de6703d9c1341cb550c056"
server_key_iv: "09b8fe7749a38851f30965c9f8c1dd20"
server_key_version: "kuduclusterkey@0"
I20260809 23:01:56.213671 7360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:01:56.258401 7360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:01:56.263700 7360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:01:56.264082 7360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:01:56.265359 7360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:01:56.267238 7360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:01:56.267390 7360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 23:01:56.267539 7360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:01:56.267656 7360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:01:56.311760 7360 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:46189
I20260809 23:01:56.311863 7514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:46189 every 8 connection(s)
I20260809 23:01:56.313328 7360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:01:56.314975 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 7360
Aug 09 23:01:56 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316516, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:01:56.366148 7520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.316483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.24.31.254:39487)
0809 23:01:56.340203 (+ 23720us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.340213 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.345916 (+ 5703us) server_negotiation.cc:407] Connection header received
0809 23:01:56.346725 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.346731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.346831 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.346919 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:56.349449 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.350521 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.352469 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.353107 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.353258 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.357343 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:56.357350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:56.357355 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:56.357392 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:56.361339 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:56.362252 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:56.362258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:56.362261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:56.362497 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:56.364130 (+ 1633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:56.364135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:56.364138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:56.364527 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:56.364781 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.365516 (+ 735us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.365733 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23524,"thread_start_us":29609,"threads_started":1}
I20260809 23:01:56.369933 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.317400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39487 (local address 127.24.31.195:56079)
0809 23:01:56.343463 (+ 26063us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:56.343547 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:56.347668 (+ 4121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:56.348869 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:56.348882 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:56.349576 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:01:56.350360 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.350394 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.351920 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.351926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.354038 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.354058 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.355492 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.355502 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.355823 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.356558 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:01:56.356601 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:01:56.359667 (+ 3066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:01:56.364074 (+ 4407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:56.364093 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:56.364116 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:56.364840 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:56.365312 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:01:56.365317 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:01:56.365320 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:01:56.366617 (+ 1297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:01:56.367373 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:01:56.367381 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:01:56.367749 (+ 368us) client_negotiation.cc:769] Sending connection context
0809 23:01:56.368087 (+ 338us) client_negotiation.cc:240] Negotiation successful
0809 23:01:56.368670 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25663,"thread_start_us":25419,"threads_started":1}
I20260809 23:01:56.370983 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.317394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:56079 (local address 127.24.31.254:39487)
0809 23:01:56.346366 (+ 28972us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.346371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.346390 (+ 19us) server_negotiation.cc:407] Connection header received
0809 23:01:56.348123 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.348130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.348233 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.348353 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:56.350614 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.351745 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.354227 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.355029 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.355151 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.359961 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:56.359968 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:56.359974 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:56.360034 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:56.363829 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:56.364975 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:56.364980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:56.364984 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:56.365116 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:56.366769 (+ 1653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:56.366775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:56.366778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:56.367123 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:56.367278 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.370568 (+ 3290us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.370721 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28854,"thread_start_us":22842,"threads_started":1}
I20260809 23:01:56.372071 7515 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39487
I20260809 23:01:56.372397 7515 heartbeater.cc:464] Registering TS with master...
I20260809 23:01:56.373106 7515 heartbeater.cc:511] Master 127.24.31.254:39487 requested a full tablet report, sending...
I20260809 23:01:56.374504 6929 ts_manager.cc:194] Registered new tserver with Master: 774f5d5c291d46f5b69f051d2d2faaf3 (127.24.31.195:46189)
I20260809 23:01:56.375303 6929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:56079
I20260809 23:01:56.381950 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:01:56.399403 7519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.385442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.24.31.254:39487)
0809 23:01:56.385659 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.385669 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.386288 (+ 619us) server_negotiation.cc:407] Connection header received
0809 23:01:56.386509 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.386515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.386651 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.386766 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:56.388324 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.389042 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.390512 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.390940 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.391076 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.393870 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:56.393876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:56.393880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:56.393924 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:56.396604 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:56.397340 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:56.397345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:56.397349 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:56.397479 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:56.397896 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:56.397901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:56.397906 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:56.398188 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:56.398449 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.398796 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.399045 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 23:01:56.404662 6929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:01:56.412362 6929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:01:56.452287 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.428729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:46189 (local address 127.0.0.1:59110)
0809 23:01:56.436980 (+ 8251us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:56.437000 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:56.438879 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:56.440334 (+ 1455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:56.440346 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:56.440402 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:56.441502 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.441516 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.444253 (+ 2737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.444262 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.447608 (+ 3346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.447633 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.450882 (+ 3249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.450890 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.451145 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.451222 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 23:01:56.451365 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 23:01:56.451644 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8131,"thread_start_us":8342,"threads_started":1}
I20260809 23:01:56.453822 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.429579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59110 (local address 127.24.31.195:46189)
0809 23:01:56.436488 (+ 6909us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.436498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.438705 (+ 2207us) server_negotiation.cc:407] Connection header received
0809 23:01:56.439165 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.439178 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.439527 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.439832 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:56.441780 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.443997 (+ 2217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.447902 (+ 3905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.450245 (+ 2343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.450405 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.450696 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.452977 (+ 2281us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.453262 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6746,"thread_start_us":6554,"threads_started":1}
I20260809 23:01:56.459311 7437 tablet_service.cc:1467] Processing CreateTablet for tablet 901760bfae3d40c5ab20a931f5961a7a (DEFAULT_TABLE table=test-table [id=7e0ca1cd74484cd990f2e0ae82c88488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:01:56.461004 7437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 901760bfae3d40c5ab20a931f5961a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:01:56.464676 7535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.427297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:39985 (local address 127.0.0.1:53792)
0809 23:01:56.435674 (+ 8377us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:56.435718 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:56.435921 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:56.444641 (+ 8720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:56.444646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:56.444678 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:56.445126 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.445139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.452481 (+ 7342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.452486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.454537 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.454554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.463791 (+ 9237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.463811 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.464012 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.464050 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:01:56.464139 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 23:01:56.464290 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8096,"thread_start_us":7970,"threads_started":1}
I20260809 23:01:56.467471 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.428034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53792 (local address 127.24.31.193:39985)
0809 23:01:56.435095 (+ 7061us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.435105 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.443334 (+ 8229us) server_negotiation.cc:407] Connection header received
0809 23:01:56.443460 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.443469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.443727 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.443931 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:56.448519 (+ 4588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.450065 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.457094 (+ 7029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.458559 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.458698 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.458933 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.466929 (+ 7996us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.467122 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6891,"thread_start_us":6698,"threads_started":1}
I20260809 23:01:56.471877 7095 tablet_service.cc:1467] Processing CreateTablet for tablet 901760bfae3d40c5ab20a931f5961a7a (DEFAULT_TABLE table=test-table [id=7e0ca1cd74484cd990f2e0ae82c88488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:01:56.474995 7095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 901760bfae3d40c5ab20a931f5961a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:01:56.484026 7540 tablet_bootstrap.cc:492] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: Bootstrap starting.
I20260809 23:01:56.489841 7540 tablet_bootstrap.cc:654] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: Neither blocks nor log segments found. Creating new log.
I20260809 23:01:56.491256 7540 log.cc:824] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:01:56.494701 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.437376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:35599 (local address 127.0.0.1:49898)
0809 23:01:56.454357 (+ 16981us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:56.454381 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:56.454523 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:56.467765 (+ 13242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:56.467770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:56.467816 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:56.467988 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.467998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.476742 (+ 8744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.476748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.479220 (+ 2472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.479256 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.493443 (+ 14187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.493455 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.493844 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.493910 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 23:01:56.494016 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 23:01:56.494251 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16869,"thread_start_us":29073,"threads_started":1}
I20260809 23:01:56.496222 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.437612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49898 (local address 127.24.31.194:35599)
0809 23:01:56.465565 (+ 27953us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.465575 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.465604 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:01:56.465702 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.465710 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.465958 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.468179 (+ 2221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:56.468409 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.469982 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.479539 (+ 9557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.481552 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.495036 (+ 13484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.495366 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.495481 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.495746 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27780,"thread_start_us":28497,"threads_started":1}
I20260809 23:01:56.499907 7542 tablet_bootstrap.cc:492] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: Bootstrap starting.
I20260809 23:01:56.500715 7274 tablet_service.cc:1467] Processing CreateTablet for tablet 901760bfae3d40c5ab20a931f5961a7a (DEFAULT_TABLE table=test-table [id=7e0ca1cd74484cd990f2e0ae82c88488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:01:56.501992 7540 tablet_bootstrap.cc:492] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: No bootstrap required, opened a new log
I20260809 23:01:56.502216 7274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 901760bfae3d40c5ab20a931f5961a7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:01:56.502300 7540 ts_tablet_manager.cc:1397] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.008s
I20260809 23:01:56.513113 7542 tablet_bootstrap.cc:654] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: Neither blocks nor log segments found. Creating new log.
I20260809 23:01:56.514612 7542 log.cc:824] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:01:56.519115 7540 raft_consensus.cc:348] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } }
I20260809 23:01:56.520984 7540 raft_consensus.cc:374] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:01:56.521524 7540 raft_consensus.cc:729] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 774f5d5c291d46f5b69f051d2d2faaf3, State: Initialized, Role: FOLLOWER
I20260809 23:01:56.523205 7540 consensus_queue.cc:260] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } }
I20260809 23:01:56.527549 7542 tablet_bootstrap.cc:492] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: No bootstrap required, opened a new log
I20260809 23:01:56.527930 7542 ts_tablet_manager.cc:1397] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: Time spent bootstrapping tablet: real 0.028s user 0.006s sys 0.004s
I20260809 23:01:56.534703 7543 tablet_bootstrap.cc:492] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: Bootstrap starting.
I20260809 23:01:56.538626 7515 heartbeater.cc:503] Master 127.24.31.254:39487 was elected leader, sending a full tablet report...
I20260809 23:01:56.539665 7540 ts_tablet_manager.cc:1428] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: Time spent starting tablet: real 0.037s user 0.008s sys 0.008s
I20260809 23:01:56.540019 7542 raft_consensus.cc:348] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } }
I20260809 23:01:56.540621 7542 raft_consensus.cc:374] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:01:56.540799 7542 raft_consensus.cc:729] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c777c600cf4d4999a538da92bc751d9b, State: Initialized, Role: FOLLOWER
I20260809 23:01:56.541819 7543 tablet_bootstrap.cc:654] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: Neither blocks nor log segments found. Creating new log.
I20260809 23:01:56.541644 7542 consensus_queue.cc:260] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } }
I20260809 23:01:56.543280 7543 log.cc:824] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:01:56.550159 7542 ts_tablet_manager.cc:1428] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: Time spent starting tablet: real 0.022s user 0.018s sys 0.001s
I20260809 23:01:56.550612 7175 heartbeater.cc:503] Master 127.24.31.254:39487 was elected leader, sending a full tablet report...
I20260809 23:01:56.559082 7543 tablet_bootstrap.cc:492] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: No bootstrap required, opened a new log
I20260809 23:01:56.559484 7543 ts_tablet_manager.cc:1397] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.013s
W20260809 23:01:56.565007 7517 tablet.cc:2369] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:01:56.570534 7543 raft_consensus.cc:348] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } }
I20260809 23:01:56.571031 7543 raft_consensus.cc:374] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:01:56.571162 7543 raft_consensus.cc:729] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f81f5a4f27a74a58ad8f51bbf266a26c, State: Initialized, Role: FOLLOWER
I20260809 23:01:56.572019 7543 consensus_queue.cc:260] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } }
I20260809 23:01:56.573407 7356 heartbeater.cc:503] Master 127.24.31.254:39487 was elected leader, sending a full tablet report...
I20260809 23:01:56.574369 7543 ts_tablet_manager.cc:1428] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
Aug 09 23:01:56 dist-test-slave-vxj2 krb5kdc[5741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316515, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for HTTP/127.24.31.212@KRBTEST.COM
W20260809 23:01:56.658783 7357 tablet.cc:2369] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:01:56.681344 7548 raft_consensus.cc:482] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:01:56.681737 7548 raft_consensus.cc:504] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } }
I20260809 23:01:56.684026 7548 leader_election.cc:288] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 774f5d5c291d46f5b69f051d2d2faaf3 (127.24.31.195:46189), f81f5a4f27a74a58ad8f51bbf266a26c (127.24.31.194:35599)
I20260809 23:01:56.702534 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.686033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:36585 (local address 127.24.31.195:46189)
0809 23:01:56.686270 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.686276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.693904 (+ 7628us) server_negotiation.cc:407] Connection header received
0809 23:01:56.694314 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.694323 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.694432 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.694511 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:56.695584 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.696683 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.700069 (+ 3386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.701313 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.701448 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.701536 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.702124 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.702268 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 23:01:56.703627 7553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.684648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:46189 (local address 127.24.31.193:36585)
0809 23:01:56.693732 (+ 9084us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:56.693761 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:56.694087 (+ 326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:56.694787 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:56.694795 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:56.694833 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:56.695385 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.695402 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.696915 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.696924 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.699850 (+ 2926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.699869 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.701729 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.701737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.701922 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.701970 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 23:01:56.703088 (+ 1118us) client_negotiation.cc:240] Negotiation successful
0809 23:01:56.703242 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8922,"thread_start_us":15365,"threads_started":1}
I20260809 23:01:56.709355 7460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "901760bfae3d40c5ab20a931f5961a7a" candidate_uuid: "c777c600cf4d4999a538da92bc751d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" is_pre_election: true
I20260809 23:01:56.710059 7460 raft_consensus.cc:2454] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c777c600cf4d4999a538da92bc751d9b in term 0.
I20260809 23:01:56.711025 7037 leader_election.cc:302] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 774f5d5c291d46f5b69f051d2d2faaf3, c777c600cf4d4999a538da92bc751d9b; no voters:
I20260809 23:01:56.711557 7548 raft_consensus.cc:2781] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:01:56.711711 7548 raft_consensus.cc:482] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:01:56.711815 7548 raft_consensus.cc:3037] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:01:56.715204 7548 raft_consensus.cc:504] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } }
I20260809 23:01:56.716733 7460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "901760bfae3d40c5ab20a931f5961a7a" candidate_uuid: "c777c600cf4d4999a538da92bc751d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "774f5d5c291d46f5b69f051d2d2faaf3"
I20260809 23:01:56.716953 7460 raft_consensus.cc:3037] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:01:56.720957 7548 leader_election.cc:288] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [CANDIDATE]: Term 1 election: Requested vote from peers 774f5d5c291d46f5b69f051d2d2faaf3 (127.24.31.195:46189), f81f5a4f27a74a58ad8f51bbf266a26c (127.24.31.194:35599)
I20260809 23:01:56.722110 7460 raft_consensus.cc:2454] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c777c600cf4d4999a538da92bc751d9b in term 1.
I20260809 23:01:56.722990 7552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.685337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:35599 (local address 127.24.31.193:50421)
0809 23:01:56.699871 (+ 14534us) negotiation.cc:107] Waiting for socket to connect
0809 23:01:56.699898 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 23:01:56.700294 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:01:56.700825 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:01:56.700830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:01:56.700857 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:01:56.701208 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.701222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.702643 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.702651 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.704618 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:01:56.704633 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.722097 (+ 17464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.722106 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:01:56.722316 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.722360 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 23:01:56.722453 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 23:01:56.722624 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14385,"thread_start_us":8040,"threads_started":1}
I20260809 23:01:56.723074 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.685293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:50421 (local address 127.24.31.194:35599)
0809 23:01:56.685670 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.685678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.699994 (+ 14316us) server_negotiation.cc:407] Connection header received
0809 23:01:56.700491 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.700500 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.700576 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.700636 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:01:56.701414 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.702454 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.704807 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.706062 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.706188 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.706256 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.722656 (+ 16400us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.722826 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260809 23:01:56.724390 7037 leader_election.cc:302] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 774f5d5c291d46f5b69f051d2d2faaf3, c777c600cf4d4999a538da92bc751d9b; no voters:
I20260809 23:01:56.724295 7300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "901760bfae3d40c5ab20a931f5961a7a" candidate_uuid: "c777c600cf4d4999a538da92bc751d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" is_pre_election: true
I20260809 23:01:56.724831 7548 raft_consensus.cc:2781] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:01:56.724884 7300 raft_consensus.cc:2454] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c777c600cf4d4999a538da92bc751d9b in term 0.
I20260809 23:01:56.725487 7299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "901760bfae3d40c5ab20a931f5961a7a" candidate_uuid: "c777c600cf4d4999a538da92bc751d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c"
I20260809 23:01:56.725706 7299 raft_consensus.cc:3037] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:01:56.728910 7299 raft_consensus.cc:2454] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c777c600cf4d4999a538da92bc751d9b in term 1.
I20260809 23:01:56.736882 7548 raft_consensus.cc:686] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 1 LEADER]: Becoming Leader. State: Replica: c777c600cf4d4999a538da92bc751d9b, State: Running, Role: LEADER
I20260809 23:01:56.737573 7548 consensus_queue.cc:237] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } }
W20260809 23:01:56.751565 7176 tablet.cc:2369] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:01:56.756021 6929 catalog_manager.cc:5495] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to c777c600cf4d4999a538da92bc751d9b (127.24.31.193). New cstate: current_term: 1 leader_uuid: "c777c600cf4d4999a538da92bc751d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c777c600cf4d4999a538da92bc751d9b" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 39985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260809 23:01:56.827492 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.818981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.24.31.193:39985)
0809 23:01:56.819208 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.819215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.819236 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:01:56.819350 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.819356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.819474 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.819655 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:01:56.820171 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.821333 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.823415 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.824043 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.824167 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.825557 (+ 1390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:01:56.826326 (+ 769us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:01:56.826497 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.827123 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.827204 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 23:01:56.860411 7299 raft_consensus.cc:1261] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [term 1 FOLLOWER]: Refusing update from remote peer c777c600cf4d4999a538da92bc751d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:01:56.860411 7460 raft_consensus.cc:1261] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [term 1 FOLLOWER]: Refusing update from remote peer c777c600cf4d4999a538da92bc751d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:01:56.861565 7548 consensus_queue.cc:1035] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f81f5a4f27a74a58ad8f51bbf266a26c" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:01:56.861928 7555 consensus_queue.cc:1035] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "774f5d5c291d46f5b69f051d2d2faaf3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:01:56.890987 7558 mvcc.cc:205] Tried to move back new op lower bound from 7316752453045784576 to 7316752452562812928. Current Snapshot: MvccSnapshot[applied={T|T < 7316752453045784576}]
I20260809 23:01:56.915979 6929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:01:56.916428 6929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:01:56.920878 6929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d31743ec2f6e49ab80de61bd6e2c7b76: Sending DeleteTablet for 3 replicas of tablet 901760bfae3d40c5ab20a931f5961a7a
I20260809 23:01:56.922144 7095 tablet_service.cc:1514] Processing DeleteTablet for tablet 901760bfae3d40c5ab20a931f5961a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:01:56 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:53792
I20260809 23:01:56.927790 7437 tablet_service.cc:1514] Processing DeleteTablet for tablet 901760bfae3d40c5ab20a931f5961a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:01:56 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:59110
I20260809 23:01:56.928499 7274 tablet_service.cc:1514] Processing DeleteTablet for tablet 901760bfae3d40c5ab20a931f5961a7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:01:56 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:49898
I20260809 23:01:56.934185 7577 tablet_replica.cc:330] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: stopping tablet replica
I20260809 23:01:56.935252 7577 raft_consensus.cc:2229] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:01:56.936622 7577 raft_consensus.cc:2258] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:01:56.938763 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 6976
I20260809 23:01:56.941787 7577 ts_tablet_manager.cc:1905] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:01:56.945999 7581 tablet_replica.cc:330] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: stopping tablet replica
I20260809 23:01:56.946528 7581 raft_consensus.cc:2229] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:01:56.947147 7581 raft_consensus.cc:2258] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:01:56.948312 7579 tablet_replica.cc:330] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: stopping tablet replica
I20260809 23:01:56.948774 7579 raft_consensus.cc:2229] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:01:56.949361 7579 raft_consensus.cc:2258] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:01:56.952751 7581 ts_tablet_manager.cc:1905] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:01:56.961009 7581 ts_tablet_manager.cc:1918] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:01:56.969301 7581 log.cc:1196] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/901760bfae3d40c5ab20a931f5961a7a
I20260809 23:01:56.969352 7577 ts_tablet_manager.cc:1918] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:01:56.970036 7581 ts_tablet_manager.cc:1939] T 901760bfae3d40c5ab20a931f5961a7a P f81f5a4f27a74a58ad8f51bbf266a26c: Deleting consensus metadata
I20260809 23:01:56.971091 7579 ts_tablet_manager.cc:1905] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:01:56.972097 6912 catalog_manager.cc:4841] TS f81f5a4f27a74a58ad8f51bbf266a26c (127.24.31.194:35599): tablet 901760bfae3d40c5ab20a931f5961a7a (table test-table [id=7e0ca1cd74484cd990f2e0ae82c88488]) successfully deleted
I20260809 23:01:56.974905 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:56.939998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53802 (local address 127.24.31.193:39985)
0809 23:01:56.940187 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:56.940195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:56.944907 (+ 4712us) server_negotiation.cc:407] Connection header received
0809 23:01:56.945511 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:56.945520 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:56.945636 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:56.945737 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:56.947806 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.950987 (+ 3181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.952859 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:56.953548 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:56.953657 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:56.957293 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:56.957307 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:56.957326 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:56.957364 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:56.961658 (+ 4294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:56.971338 (+ 9680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:56.971348 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:56.971352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:56.971556 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:56.972074 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:56.972079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:56.972083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:56.972526 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:56.972828 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:56.974213 (+ 1385us) server_negotiation.cc:299] Negotiation successful
0809 23:01:56.974495 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 23:01:56.982414 7170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:56.983480 7577 log.cc:1196] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/901760bfae3d40c5ab20a931f5961a7a
I20260809 23:01:56.984633 7579 ts_tablet_manager.cc:1918] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:01:56.984894 7579 log.cc:1196] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/901760bfae3d40c5ab20a931f5961a7a
I20260809 23:01:56.985628 7579 ts_tablet_manager.cc:1939] T 901760bfae3d40c5ab20a931f5961a7a P 774f5d5c291d46f5b69f051d2d2faaf3: Deleting consensus metadata
I20260809 23:01:56.988170 6913 catalog_manager.cc:4841] TS 774f5d5c291d46f5b69f051d2d2faaf3 (127.24.31.195:46189): tablet 901760bfae3d40c5ab20a931f5961a7a (table test-table [id=7e0ca1cd74484cd990f2e0ae82c88488]) successfully deleted
I20260809 23:01:57.090221 7577 ts_tablet_manager.cc:1939] T 901760bfae3d40c5ab20a931f5961a7a P c777c600cf4d4999a538da92bc751d9b: Deleting consensus metadata
I20260809 23:01:57.092376 6912 catalog_manager.cc:4841] TS c777c600cf4d4999a538da92bc751d9b (127.24.31.193:39985): tablet 901760bfae3d40c5ab20a931f5961a7a (table test-table [id=7e0ca1cd74484cd990f2e0ae82c88488]) successfully deleted
I20260809 23:01:57.101814 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 6976
I20260809 23:01:57.125094 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 7180
I20260809 23:01:57.140261 7539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:57.125969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49910 (local address 127.24.31.194:35599)
0809 23:01:57.126158 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:57.126164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:57.126315 (+ 151us) server_negotiation.cc:407] Connection header received
0809 23:01:57.126546 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:57.126551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:57.126615 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:57.126722 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:57.128066 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:57.128781 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:57.130518 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:57.130969 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:57.131100 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:57.134158 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:57.134183 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:57.134198 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:57.134234 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:57.137207 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:57.137925 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:57.137935 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:57.137939 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:57.138113 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:57.138486 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:57.138492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:57.138501 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:57.138946 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:57.139239 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:57.139596 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 23:01:57.139847 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 23:01:57.140925 7351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:57.247663 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 7180
I20260809 23:01:57.267233 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 7360
I20260809 23:01:57.281936 7586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:01:57.268086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.24.31.195:46189)
0809 23:01:57.268584 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0809 23:01:57.268589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:01:57.268610 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:01:57.268675 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:01:57.268679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:01:57.268739 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:01:57.268837 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:01:57.270671 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:57.271383 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:57.272731 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:01:57.273140 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:01:57.273349 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:01:57.275921 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:01:57.275937 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:01:57.275952 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:01:57.275983 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:01:57.278965 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:57.279740 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:57.279748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:57.279750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:57.279882 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:01:57.280288 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:01:57.280295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:01:57.280303 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:01:57.280663 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:01:57.280980 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 23:01:57.281338 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 23:01:57.281565 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":266,"threads_started":1}
I20260809 23:01:57.282578 7510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:57.371943 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 7360
I20260809 23:01:57.393496 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 6891
I20260809 23:01:57.394707 6969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:01:57.506067 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 6891
I20260809 23:01:57.530928 24703 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 23:01:57.876896 24703 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 23:01:57.877102 24703 mini_ranger.cc:66] Stopping Ranger...
I20260809 23:01:58.222635 24703 mini_ranger.cc:68] Stopped Ranger
2026-08-09 23:01:58.222 UTC [5777] LOG: received smart shutdown request
2026-08-09 23:01:58.225 UTC [5777] LOG: background worker "logical replication launcher" (PID 5786) exited with exit code 1
2026-08-09 23:01:58.226 UTC [5781] LOG: shutting down
2026-08-09 23:01:58.394 UTC [5777] LOG: database system is shut down
2026-08-09T23:01:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54304 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:01:58 dist-test-slave-vxj2 krb5kdc[7598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:01:58 dist-test-slave-vxj2 krb5kdc[7598](info): set up 2 sockets
Aug 09 23:01:58 dist-test-slave-vxj2 krb5kdc[7598](info): commencing operation
krb5kdc: starting...
W20260809 23:01:59.728924 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:01:59 dist-test-slave-vxj2 krb5kdc[7598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:01:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:01:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:01:59.977113 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:34273
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:35855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:34273,127.24.31.253:44765,127.24.31.252:46127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:00.316874 7615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:00.317238 7615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:00.317369 7615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:00.324518 7615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:02:00.324611 7615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:00.324658 7615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:00.324709 7615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:02:00.324745 7615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:02:00.334703 7615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:35855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:34273,127.24.31.253:44765,127.24.31.252:46127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:34273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:00.336400 7615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:00.337716 7615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:00.349762 7621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:00.349737 7623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:00.350030 7625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:00.350483 7615 server_base.cc:1034] running on GCE node
I20260809 23:02:00.351322 7615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:00.352612 7615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:00.353859 7615 hybrid_clock.cc:648] HybridClock initialized: now 1786316520353808 us; error 57 us; skew 500 ppm
I20260809 23:02:00.354228 7615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:00 dist-test-slave-vxj2 krb5kdc[7598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:00.361379 7615 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:02:00.362836 7615 webserver.cc:460] Webserver started at http://127.24.31.254:35035/ using document root <none> and password file <none>
I20260809 23:02:00.363541 7615 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:00.363692 7615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:00.364004 7615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:00.367100 7615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "90fe5278f02e4621aabca0cfbb05c67c"
format_stamp: "Formatted at 2026-08-09 23:02:00 on dist-test-slave-vxj2"
server_key: "7f7ef3814b9a3e22d04253b92e1961d3"
server_key_iv: "fd6104f3f8205cf708c9e6baf1180d4b"
server_key_version: "encryptionkey@0"
I20260809 23:02:00.367920 7615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "90fe5278f02e4621aabca0cfbb05c67c"
format_stamp: "Formatted at 2026-08-09 23:02:00 on dist-test-slave-vxj2"
server_key: "7f7ef3814b9a3e22d04253b92e1961d3"
server_key_iv: "fd6104f3f8205cf708c9e6baf1180d4b"
server_key_version: "encryptionkey@0"
I20260809 23:02:00.372921 7615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 23:02:00.376259 7637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:00.377605 7615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:00.377811 7615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "90fe5278f02e4621aabca0cfbb05c67c"
format_stamp: "Formatted at 2026-08-09 23:02:00 on dist-test-slave-vxj2"
server_key: "7f7ef3814b9a3e22d04253b92e1961d3"
server_key_iv: "fd6104f3f8205cf708c9e6baf1180d4b"
server_key_version: "encryptionkey@0"
I20260809 23:02:00.378016 7615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:00.405383 7615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:00.410215 7615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:00.410591 7615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:00.430871 7615 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:34273
I20260809 23:02:00.430888 7693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:34273 every 8 connection(s)
I20260809 23:02:00.432510 7615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:02:00.436259 7695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:00.442632 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 7615
I20260809 23:02:00.442898 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:02:00.443588 24703 external_mini_cluster.cc:1428] Setting key 5554d9ab61b01408fa68799304334bf9
I20260809 23:02:00.442868 7695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:02:00 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:00.468209 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.446131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:34273 (local address 127.0.0.1:47212)
0809 23:02:00.446997 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.447061 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:00.449936 (+ 2875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:00.450802 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:00.450812 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:00.451613 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:00.452169 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:00.452207 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.454181 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.454185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:00.455330 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:00.455347 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.456718 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.456723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:00.456862 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:00.457553 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:00.457579 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:00.459755 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:00.464117 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:00.464127 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:00.464137 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:00.464559 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:00.465060 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:00.465064 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:00.465066 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:00.465288 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:00.466352 (+ 1064us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:00.466358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:00.466574 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 23:02:00.466784 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 23:02:00.467164 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":322,"threads_started":1}
I20260809 23:02:00.468250 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.447188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.24.31.254:34273)
0809 23:02:00.447820 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:00.447826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:00.447869 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:02:00.450305 (+ 2436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:00.450311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:00.450546 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:00.450711 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:00.452374 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.454005 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.455497 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.456491 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.456688 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:00.459993 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:00.460006 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:00.460025 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:00.460072 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:00.463966 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:00.464695 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:00.464706 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:00.464712 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:00.464944 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:00.465395 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:00.465403 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:00.465409 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:00.466182 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:00.466456 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:00.466992 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 23:02:00.467365 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":454,"thread_start_us":310,"threads_started":1}
WARNING: no policy specified for kudu/127.24.31.253@KRBTEST.COM; defaulting to no policy
I20260809 23:02:00.471535 7695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:02:00.472529 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.472166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57234)
0809 23:02:00.472323 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.472425 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 23:02:00.472781 7642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.253@KRBTEST.COM" created.
I20260809 23:02:00.474570 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.474138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57246)
0809 23:02:00.474305 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.474391 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 23:02:00.474790 7695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:44765: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab.
Entry for principal kudu/127.24.31.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab.
I20260809 23:02:00.504207 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.253:44765
--webserver_interface=127.24.31.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab
--principal=kudu/127.24.31.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:35855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:34273,127.24.31.253:44765,127.24.31.252:46127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:02:00.524147 7695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } attempt: 1
I20260809 23:02:00.525913 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.525033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57258)
0809 23:02:00.525452 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.525644 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260809 23:02:00.527172 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.526790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57272)
0809 23:02:00.526977 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.527047 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 23:02:00.527357 7695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:44765: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
I20260809 23:02:00.595645 7695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } attempt: 2
I20260809 23:02:00.596992 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.596432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57286)
0809 23:02:00.596710 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.596836 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 23:02:00.598126 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.597750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57298)
0809 23:02:00.597926 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.598005 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 23:02:00.598323 7695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:44765: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
I20260809 23:02:00.689566 7695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } attempt: 3
I20260809 23:02:00.690788 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.690329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57300)
0809 23:02:00.690544 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.690640 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 23:02:00.691901 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.691495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57302)
0809 23:02:00.691650 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.691773 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 23:02:00.692086 7695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:44765: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
W20260809 23:02:00.832752 7703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:00.833092 7703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:00.833289 7703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 23:02:00.835335 7695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } attempt: 4
I20260809 23:02:00.836642 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.836017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57318)
0809 23:02:00.836383 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.836469 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 23:02:00.837702 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.837356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57326)
0809 23:02:00.837500 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.837590 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 23:02:00.837941 7695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:44765: Network error: Client connection negotiation failed: client connection to 127.24.31.253:44765: connect: Connection refused (error 111)
W20260809 23:02:00.840786 7703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:02:00.840919 7703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:00.841020 7703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:00.841121 7703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:02:00.841223 7703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:02:00.850318 7703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:35855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:34273,127.24.31.253:44765,127.24.31.252:46127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab
--principal=kudu/127.24.31.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.253:44765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.31.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:00.851548 7703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:00.852859 7703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:00.864609 7710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:00.864710 7709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:00.864964 7712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:00.865607 7703 server_base.cc:1034] running on GCE node
I20260809 23:02:00.866333 7703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:00.867676 7703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:00.868912 7703 hybrid_clock.cc:648] HybridClock initialized: now 1786316520868880 us; error 58 us; skew 500 ppm
I20260809 23:02:00.869326 7703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:00 dist-test-slave-vxj2 krb5kdc[7598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:00.872395 7703 init.cc:377] Logged in from keytab as kudu/127.24.31.253@KRBTEST.COM (short username kudu)
I20260809 23:02:00.873991 7703 webserver.cc:460] Webserver started at http://127.24.31.253:42381/ using document root <none> and password file <none>
I20260809 23:02:00.874670 7703 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:00.874817 7703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:00.875142 7703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:00.878094 7703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/data/instance:
uuid: "c14d0c97d10e4737acb92a280c143ed1"
format_stamp: "Formatted at 2026-08-09 23:02:00 on dist-test-slave-vxj2"
server_key: "3aa09f8191e5d943e27b8388db236b8d"
server_key_iv: "bf3c5001536735875bcba3912adf56af"
server_key_version: "encryptionkey@0"
I20260809 23:02:00.878887 7703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/wal/instance:
uuid: "c14d0c97d10e4737acb92a280c143ed1"
format_stamp: "Formatted at 2026-08-09 23:02:00 on dist-test-slave-vxj2"
server_key: "3aa09f8191e5d943e27b8388db236b8d"
server_key_iv: "bf3c5001536735875bcba3912adf56af"
server_key_version: "encryptionkey@0"
I20260809 23:02:00.884027 7703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 23:02:00.887458 7721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:00.888796 7703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 23:02:00.888998 7703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/wal
uuid: "c14d0c97d10e4737acb92a280c143ed1"
format_stamp: "Formatted at 2026-08-09 23:02:00 on dist-test-slave-vxj2"
server_key: "3aa09f8191e5d943e27b8388db236b8d"
server_key_iv: "bf3c5001536735875bcba3912adf56af"
server_key_version: "encryptionkey@0"
I20260809 23:02:00.889241 7703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:00.902348 7703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:00.906214 7703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:00.906512 7703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:00.924538 7703 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.253:44765
I20260809 23:02:00.924577 7783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.253:44765 every 8 connection(s)
I20260809 23:02:00.925864 7703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
I20260809 23:02:00.928328 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 7703
I20260809 23:02:00.928521 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-1/wal/instance
I20260809 23:02:00.929102 24703 external_mini_cluster.cc:1428] Setting key 108ab5abbbcff369c851a9a2f10941a7
I20260809 23:02:00.929883 7785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:00.939634 7785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:02:00 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.31.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.252@KRBTEST.COM" created.
I20260809 23:02:00.960760 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.942458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.24.31.254:34273)
0809 23:02:00.942674 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:00.942681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:00.943828 (+ 1147us) server_negotiation.cc:407] Connection header received
0809 23:02:00.945880 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:00.945889 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:00.945992 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:00.946116 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:00.947539 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.948647 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.950194 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.950960 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.951097 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:00.954555 (+ 3458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:00.954575 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:00.954580 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:00.954638 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:00.957535 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:00.958516 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:00.958520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:00.958522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:00.958636 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:00.959134 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:00.959137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:00.959139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:00.959401 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:00.959520 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:00.960351 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 23:02:00.960535 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 23:02:00.962296 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.942835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:34273 (local address 127.0.0.1:47216)
0809 23:02:00.943644 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.943696 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:00.945614 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:00.946293 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:00.946300 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:00.946812 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:00.947332 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:00.947359 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.948796 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.948799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:00.950054 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:00.950066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.951184 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.951188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:00.951384 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:00.952106 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:00.952128 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:00.954322 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:00.957707 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:00.957713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:00.957724 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:00.958338 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:00.958743 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:00.958751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:00.958757 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:00.959018 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:00.959534 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:00.959545 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:00.959981 (+ 436us) client_negotiation.cc:769] Sending connection context
0809 23:02:00.960275 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 23:02:00.960813 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":368,"threads_started":1}
I20260809 23:02:00.964035 7785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:02:00 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
I20260809 23:02:00.978618 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.964693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57334)
0809 23:02:00.964942 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.964965 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:00.965064 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:00.965936 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:00.965940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:00.966359 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:00.966501 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:00.966527 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.968499 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.968506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:00.969430 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:00.969443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.970244 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.970250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:00.970461 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:00.971038 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:00.971053 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:00.973280 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:00.976211 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:00.976217 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:00.976224 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:00.976574 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:00.976933 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:00.976939 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:00.976942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:00.977104 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:00.977701 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:00.977709 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:00.977835 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 23:02:00.977955 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 23:02:00.978233 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260809 23:02:00.978698 7792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.964901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.24.31.253:44765)
0809 23:02:00.965568 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:00.965574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:00.965598 (+ 24us) server_negotiation.cc:407] Connection header received
0809 23:02:00.965641 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:00.965646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:00.965794 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:00.965959 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:00.966723 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.968362 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.969566 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:00.970093 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:00.970226 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:00.973508 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:00.973519 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:00.973530 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:00.973557 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:00.976081 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:00.976712 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:00.976718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:00.976720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:00.976827 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:00.977251 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:00.977258 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:00.977261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:00.977570 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:00.977768 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:00.978130 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 23:02:00.978356 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":394,"threads_started":1}
I20260809 23:02:00.981223 7785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:02:00.982604 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.982034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45740)
0809 23:02:00.982221 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.982368 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 23:02:00.982901 7724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
I20260809 23:02:00.984470 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:00.984108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45744)
0809 23:02:00.984242 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:00.984319 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 23:02:00.984710 7785 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:46127: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab.
Entry for principal kudu/127.24.31.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab.
I20260809 23:02:00.988498 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.252:46127
--webserver_interface=127.24.31.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab
--principal=kudu/127.24.31.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:35855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:34273,127.24.31.253:44765,127.24.31.252:46127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:02:01.034078 7785 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } attempt: 1
I20260809 23:02:01.035333 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.034838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45760)
0809 23:02:01.035057 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.035178 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 23:02:01.036417 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.036079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45776)
0809 23:02:01.036234 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.036303 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 23:02:01.036655 7785 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:46127: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
I20260809 23:02:01.104995 7785 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } attempt: 2
I20260809 23:02:01.106262 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.105782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45786)
0809 23:02:01.105999 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.106098 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 23:02:01.107328 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.106982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45788)
0809 23:02:01.107132 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.107213 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 23:02:01.107576 7785 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:46127: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
I20260809 23:02:01.137351 7695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } attempt: 5
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
I20260809 23:02:01.151233 7792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.138321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57336 (local address 127.24.31.253:44765)
0809 23:02:01.138520 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.138527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.138600 (+ 73us) server_negotiation.cc:407] Connection header received
0809 23:02:01.138805 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.138817 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.138899 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.139010 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.139820 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.140640 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.141825 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.142293 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.142431 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.145870 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.145875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.145878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.145909 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.148565 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.149262 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.149269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.149272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.149402 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.149793 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.149800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.149808 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.150092 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.150255 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.150672 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.150887 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 23:02:01.151234 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.138247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57336)
0809 23:02:01.138485 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.138508 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:01.138665 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:01.139053 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:01.139060 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:01.139466 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:01.139651 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.139661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.140779 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.140786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.141665 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.141685 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.142446 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.142452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.142642 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.143274 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:01.143290 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:01.145677 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:01.148705 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.148712 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.148716 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.149101 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.149522 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.149527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.149531 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.149673 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.150220 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:01.150224 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:01.150380 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 23:02:01.150555 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 23:02:01.150849 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 23:02:01.152832 7695 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:02:01.153980 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.153591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45794)
0809 23:02:01.153757 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.153840 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 23:02:01.154947 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.154621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45800)
0809 23:02:01.154781 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.154847 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 23:02:01.155172 7695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:46127: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
I20260809 23:02:01.198796 7785 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } attempt: 3
I20260809 23:02:01.199973 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.199469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45816)
0809 23:02:01.199705 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.199802 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 23:02:01.201046 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.200615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45832)
0809 23:02:01.200856 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.200928 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260809 23:02:01.201282 7785 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:46127: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
I20260809 23:02:01.206372 7695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } attempt: 1
I20260809 23:02:01.207461 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.206914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45840)
0809 23:02:01.207143 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.207249 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 23:02:01.208612 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.208161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45852)
0809 23:02:01.208337 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.208461 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 23:02:01.208863 7695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:46127: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
I20260809 23:02:01.277247 7695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } attempt: 2
I20260809 23:02:01.278683 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.277982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45868)
0809 23:02:01.278304 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.278448 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 23:02:01.279943 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.279472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45874)
0809 23:02:01.279655 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.279782 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 23:02:01.280226 7695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:46127: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
W20260809 23:02:01.334725 7795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:01.335044 7795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:01.335208 7795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:01.342702 7795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:02:01.342851 7795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:01.342942 7795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:01.343053 7795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:02:01.343143 7795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:02:01.344468 7785 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } attempt: 4
I20260809 23:02:01.345633 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.345104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45884)
0809 23:02:01.345385 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.345475 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 23:02:01.346598 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.346274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45890)
0809 23:02:01.346418 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.346502 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 23:02:01.346836 7785 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:46127: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
I20260809 23:02:01.354092 7795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:35855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:34273,127.24.31.253:44765,127.24.31.252:46127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab
--principal=kudu/127.24.31.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.252:46127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.31.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:01.355501 7795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:01.356892 7795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:01.369235 7800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:01.369251 7801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:01.369935 7804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:01.370612 7795 server_base.cc:1034] running on GCE node
I20260809 23:02:01.371456 7795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:01.372787 7795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:01.374027 7795 hybrid_clock.cc:648] HybridClock initialized: now 1786316521373978 us; error 53 us; skew 500 ppm
I20260809 23:02:01.374413 7795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316521, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:01.377611 7795 init.cc:377] Logged in from keytab as kudu/127.24.31.252@KRBTEST.COM (short username kudu)
I20260809 23:02:01.379415 7795 webserver.cc:460] Webserver started at http://127.24.31.252:45495/ using document root <none> and password file <none>
I20260809 23:02:01.380242 7795 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:01.380403 7795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:01.380709 7795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:01.385677 7795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/data/instance:
uuid: "a749bca384374a64897f5b80d44460be"
format_stamp: "Formatted at 2026-08-09 23:02:01 on dist-test-slave-vxj2"
server_key: "e4445d4512ad7d4ed5bee2965db7a972"
server_key_iv: "ce70a627aaa45da11c953422c29dff01"
server_key_version: "encryptionkey@0"
I20260809 23:02:01.386471 7695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } attempt: 3
I20260809 23:02:01.386752 7795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/wal/instance:
uuid: "a749bca384374a64897f5b80d44460be"
format_stamp: "Formatted at 2026-08-09 23:02:01 on dist-test-slave-vxj2"
server_key: "e4445d4512ad7d4ed5bee2965db7a972"
server_key_iv: "ce70a627aaa45da11c953422c29dff01"
server_key_version: "encryptionkey@0"
I20260809 23:02:01.387789 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.387193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45896)
0809 23:02:01.387466 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.387568 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 23:02:01.388903 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.388432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45900)
0809 23:02:01.388660 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.388754 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260809 23:02:01.389117 7695 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:46127: Network error: Client connection negotiation failed: client connection to 127.24.31.252:46127: connect: Connection refused (error 111)
I20260809 23:02:01.392416 7795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 23:02:01.395895 7811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:01.397248 7795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:01.397467 7795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/wal
uuid: "a749bca384374a64897f5b80d44460be"
format_stamp: "Formatted at 2026-08-09 23:02:01 on dist-test-slave-vxj2"
server_key: "e4445d4512ad7d4ed5bee2965db7a972"
server_key_iv: "ce70a627aaa45da11c953422c29dff01"
server_key_version: "encryptionkey@0"
I20260809 23:02:01.397663 7795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:01.410593 7795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:01.414307 7795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:01.414614 7795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:01.433986 7795 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.252:46127
I20260809 23:02:01.434007 7873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.252:46127 every 8 connection(s)
I20260809 23:02:01.435325 7795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
I20260809 23:02:01.439036 7874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:01.443744 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 7795
I20260809 23:02:01.443936 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316311377261-24703-0/minicluster-data/master-2/wal/instance
I20260809 23:02:01.444383 24703 external_mini_cluster.cc:1428] Setting key ce6e776f38875764ff94c8bc779d8358
I20260809 23:02:01.445480 7874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316521, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
I20260809 23:02:01.464102 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.448303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.24.31.254:34273)
0809 23:02:01.448459 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.448463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.449297 (+ 834us) server_negotiation.cc:407] Connection header received
0809 23:02:01.449721 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.449726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.449806 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.449913 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.451119 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.451846 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.453487 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.453908 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.454040 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.457630 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.457637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.457644 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.457691 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.460889 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.461738 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.461744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.461747 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.461887 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.462339 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.462344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.462348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.462616 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.462782 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.463499 (+ 717us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.463733 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:02:01.465121 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.446228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45910 (local address 127.24.31.252:46127)
0809 23:02:01.447018 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.447030 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.447068 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:02:01.449483 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.449502 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.449717 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.449916 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.451405 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.452571 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.453991 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.454518 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.454657 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.459196 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.459204 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.459216 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.459244 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.462037 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.462747 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.462753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.462755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.462882 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.463300 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.463311 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.463313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.463654 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.463988 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.464283 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.464520 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":402,"threads_started":1}
I20260809 23:02:01.465140 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.448361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:34273 (local address 127.0.0.1:47222)
0809 23:02:01.449103 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.449140 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:01.449505 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:01.449972 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:01.449979 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:01.450584 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:01.450957 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.450973 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.452026 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.452030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.453337 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.453349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.454147 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.454154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.454369 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.455094 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:01.455115 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:01.457386 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:01.461028 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.461040 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.461051 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.461609 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.462007 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.462012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.462015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.462215 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.462748 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:01.462756 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:01.463097 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 23:02:01.463376 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 23:02:01.463841 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":394,"threads_started":1}
I20260809 23:02:01.466795 7874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316521, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:01.480886 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.467547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:44765 (local address 127.0.0.1:57338)
0809 23:02:01.467981 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.467996 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:01.468131 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:01.468492 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:01.468496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:01.468995 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:01.469223 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.469237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.470174 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.470182 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.471144 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.471155 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.471854 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.471860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.472080 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.472603 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:01.472626 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:01.474880 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:01.478486 (+ 3606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.478492 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.478497 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.478932 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.479342 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.479345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.479347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.479554 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.480213 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:01.480216 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:01.480355 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 23:02:01.480465 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 23:02:01.480646 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335}
I20260809 23:02:01.481381 7792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.467692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.24.31.253:44765)
0809 23:02:01.467870 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.467897 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.468095 (+ 198us) server_negotiation.cc:407] Connection header received
0809 23:02:01.468272 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.468279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.468360 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.468471 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.469367 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.470034 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.471306 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.471713 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.471804 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.475072 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.475076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.475079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.475110 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.478329 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.479055 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.479061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.479065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.479223 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.479684 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.479691 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.479696 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.480062 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.480238 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.480596 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.480818 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 23:02:01.482249 7874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:02:01.484504 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.468300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47238 (local address 127.24.31.254:34273)
0809 23:02:01.468450 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.468454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.468818 (+ 364us) server_negotiation.cc:407] Connection header received
0809 23:02:01.468982 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.468985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.469034 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.469106 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.470400 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.471119 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.472612 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.473009 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.473100 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.477773 (+ 4673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.477782 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.477787 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.477825 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.481202 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.482126 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.482131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.482134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.482282 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.482682 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.482687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.482690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.483115 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.483398 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.483963 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.484177 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316521, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
I20260809 23:02:01.496783 7880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.483045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45912)
0809 23:02:01.483217 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.483235 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:01.483331 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:01.483594 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:01.483596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:01.484009 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:01.484175 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.484186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.485643 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.485648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.486735 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.486747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.487448 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.487454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.487626 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.488230 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:01.488255 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:01.490526 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:01.494216 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.494219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.494221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.494559 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.495017 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.495024 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.495029 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.495211 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.495793 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:01.495798 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:01.495931 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 23:02:01.496075 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 23:02:01.496380 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 23:02:01.496810 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.483192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45912 (local address 127.24.31.252:46127)
0809 23:02:01.483341 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.483347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.483384 (+ 37us) server_negotiation.cc:407] Connection header received
0809 23:02:01.483432 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.483437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.483501 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.483591 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.484371 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.485458 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.486870 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.487290 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.487417 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.490699 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.490704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.490707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.490732 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.494070 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.494682 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.494688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.494692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.494852 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.495352 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.495355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.495357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.495680 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.495827 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.496181 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.496502 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":34}
I20260809 23:02:01.502070 7792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.486368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.24.31.253:44765)
0809 23:02:01.486559 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.486564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.486582 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:02:01.486666 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.486670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.486738 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.486842 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.488209 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.489366 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.490628 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.491255 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.491392 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.496475 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.496479 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.496482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.496511 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.499256 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.499927 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.499934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.499938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.500079 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.500455 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.500460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.500463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.500778 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.501067 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.501400 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.501632 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 23:02:01.502323 7874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be: Bootstrap starting.
I20260809 23:02:01.506340 7874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:01.507354 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:02:01.507514 7874 log.cc:824] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:01.510828 7874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be: No bootstrap required, opened a new log
I20260809 23:02:01.522320 7874 raft_consensus.cc:348] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } }
I20260809 23:02:01.522811 7874 raft_consensus.cc:374] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:01.522949 7874 raft_consensus.cc:729] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a749bca384374a64897f5b80d44460be, State: Initialized, Role: FOLLOWER
I20260809 23:02:01.524344 7874 consensus_queue.cc:260] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } }
I20260809 23:02:01.533371 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.513328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47250 (local address 127.24.31.254:34273)
0809 23:02:01.513515 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.513521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.514324 (+ 803us) server_negotiation.cc:407] Connection header received
0809 23:02:01.514587 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.514594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.514674 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.514747 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.517229 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.518355 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.520630 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.521320 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.521418 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.525628 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.525638 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.525644 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.525704 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.529450 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.530400 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.530405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.530409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.530579 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.531130 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.531137 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.531141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.531556 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.531826 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.532498 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.532754 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 23:02:01.536494 7890 sys_catalog.cc:455] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:01.537034 7890 sys_catalog.cc:458] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:02:01.538339 7874 sys_catalog.cc:564] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:02:01.542544 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.513122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.24.31.252:46127)
0809 23:02:01.513675 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.513680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.523931 (+ 10251us) server_negotiation.cc:407] Connection header received
0809 23:02:01.524229 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.524237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.524331 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.524405 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.526695 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.527868 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.529899 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.530652 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.530745 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.534366 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.534372 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.534376 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.534415 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.538074 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.538955 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.538963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.538970 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.539172 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.539624 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.539896 (+ 272us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.539901 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.540381 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.541806 (+ 1425us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.541916 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.542083 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419}
I20260809 23:02:01.545656 7792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.514485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57362 (local address 127.24.31.253:44765)
0809 23:02:01.514702 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.514710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.525542 (+ 10832us) server_negotiation.cc:407] Connection header received
0809 23:02:01.525729 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.525735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.525826 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.525942 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.528369 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.529871 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.532683 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.533578 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.533664 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.537896 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.537903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.537908 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.537961 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.541644 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.542549 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.542558 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.542564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.542830 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.543393 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.543400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.543405 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.543833 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.544143 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.544991 (+ 848us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.545273 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 23:02:01.549523 7905 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:02:01.549721 7905 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:02:01.546864 7795 master_runner.cc:186] Error getting master registration for 127.24.31.254:34273: OK, instance_id { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" instance_seqno: 1786316520425102 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:02:01.558305 7795 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:02:01.566363 7695 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } attempt: 4
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
I20260809 23:02:01.580992 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.567374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45940 (local address 127.24.31.252:46127)
0809 23:02:01.567576 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.567584 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.567607 (+ 23us) server_negotiation.cc:407] Connection header received
0809 23:02:01.567669 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.567677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.567753 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.567861 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.569023 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.569906 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.571122 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.571607 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.571740 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.575065 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.575070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.575073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.575125 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.578048 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.578753 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.578762 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.578768 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.578980 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.579463 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.579468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.579472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.579914 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.580073 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.580548 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.580726 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 23:02:01.581142 7697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.566961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45940)
0809 23:02:01.567178 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.567199 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:01.567324 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:01.568076 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:01.568081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:01.568634 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:01.568864 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.568879 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.570047 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.570054 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.570956 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.570968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.571759 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.571765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.571956 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.572578 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:01.572595 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:01.574866 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:01.578180 (+ 3314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.578186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.578190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.578578 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.579128 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.579133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.579136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.579324 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.580065 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:01.580070 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:01.580274 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 23:02:01.580458 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 23:02:01.580762 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 23:02:01.587347 7695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c: Bootstrap starting.
I20260809 23:02:01.591701 7695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:01.593132 7695 log.cc:824] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:01.596436 7695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c: No bootstrap required, opened a new log
I20260809 23:02:01.607491 7695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } }
I20260809 23:02:01.607918 7695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:01.608029 7695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90fe5278f02e4621aabca0cfbb05c67c, State: Initialized, Role: FOLLOWER
I20260809 23:02:01.608697 7695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } }
I20260809 23:02:01.609797 7908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:01.610147 7908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:02:01.611119 7695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [sys.catalog]: configured and running, proceeding with master startup.
W20260809 23:02:01.623194 7923 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:02:01.623411 7923 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:02:01.622012 7615 master_runner.cc:186] Error getting master registration for 127.24.31.253:44765: OK, instance_id { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" instance_seqno: 1786316520919452 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:02:01.631209 7615 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:02:01.646062 7785 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } attempt: 5
Aug 09 23:02:01 dist-test-slave-vxj2 krb5kdc[7598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316520, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
I20260809 23:02:01.659638 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.646681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:46127 (local address 127.0.0.1:45956)
0809 23:02:01.646940 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:01.646962 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:01.647071 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:01.647426 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:01.647431 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:01.647882 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:01.648038 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.648047 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.649095 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.649100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.650053 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:01.650065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.650818 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.650824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:01.651042 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.651669 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:01.651686 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:01.653817 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:01.657021 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.657027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.657031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.657507 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.657899 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:01.657904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:01.657908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:01.658055 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:01.658590 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:01.658595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:01.658781 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 23:02:01.658932 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 23:02:01.659237 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260809 23:02:01.660008 7878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:01.646841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45956 (local address 127.24.31.252:46127)
0809 23:02:01.647031 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:01.647036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:01.647054 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:02:01.647214 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:01.647219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:01.647289 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:01.647394 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:01.648188 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.648943 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.650216 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:01.650665 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:01.650808 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:01.654031 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:01.654036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:01.654039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:01.654070 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:01.656871 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.657644 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.657648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.657650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.657784 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:01.658181 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:01.658185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:01.658187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:01.658463 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:01.658641 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:01.659054 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 23:02:01.659297 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 23:02:01.663620 7644 tablet.cc:2369] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:02:01.665995 7785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1: Bootstrap starting.
W20260809 23:02:01.666641 7817 tablet.cc:2369] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:02:01.670042 7785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:01.671108 7785 log.cc:824] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:01.673848 7785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1: No bootstrap required, opened a new log
I20260809 23:02:01.681119 7785 raft_consensus.cc:348] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } }
I20260809 23:02:01.681519 7785 raft_consensus.cc:374] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:01.681715 7785 raft_consensus.cc:729] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14d0c97d10e4737acb92a280c143ed1, State: Initialized, Role: FOLLOWER
I20260809 23:02:01.682449 7785 consensus_queue.cc:260] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } }
I20260809 23:02:01.688796 7926 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:01.689311 7926 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:02:01.690150 7785 sys_catalog.cc:564] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:02:01.713367 7703 master_runner.cc:263] No leader master found from master c14d0c97d10e4737acb92a280c143ed1
I20260809 23:02:01.713553 7703 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 23:02:01.719378 7939 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:02:01.719686 7939 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 23:02:01.909235 7727 tablet.cc:2369] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:02:01.913231 7890 raft_consensus.cc:482] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:01.913590 7890 raft_consensus.cc:504] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } }
I20260809 23:02:01.915369 7890 leader_election.cc:288] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90fe5278f02e4621aabca0cfbb05c67c (127.24.31.254:34273), c14d0c97d10e4737acb92a280c143ed1 (127.24.31.253:44765)
I20260809 23:02:01.918165 7667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a749bca384374a64897f5b80d44460be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90fe5278f02e4621aabca0cfbb05c67c" is_pre_election: true
I20260809 23:02:01.916965 7754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a749bca384374a64897f5b80d44460be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14d0c97d10e4737acb92a280c143ed1" is_pre_election: true
I20260809 23:02:01.920662 7754 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a749bca384374a64897f5b80d44460be in term 0.
I20260809 23:02:01.921649 7815 leader_election.cc:302] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a749bca384374a64897f5b80d44460be, c14d0c97d10e4737acb92a280c143ed1; no voters:
I20260809 23:02:01.918825 7667 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a749bca384374a64897f5b80d44460be in term 0.
I20260809 23:02:01.923152 7890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:02:01.923322 7890 raft_consensus.cc:482] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:02:01.923420 7890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:01.926957 7890 raft_consensus.cc:504] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } }
I20260809 23:02:01.928283 7667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a749bca384374a64897f5b80d44460be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90fe5278f02e4621aabca0cfbb05c67c"
I20260809 23:02:01.928527 7667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:01.932242 7667 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a749bca384374a64897f5b80d44460be in term 1.
I20260809 23:02:01.932803 7815 leader_election.cc:302] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90fe5278f02e4621aabca0cfbb05c67c, a749bca384374a64897f5b80d44460be; no voters:
I20260809 23:02:01.934849 7890 leader_election.cc:288] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [CANDIDATE]: Term 1 election: Requested vote from peers 90fe5278f02e4621aabca0cfbb05c67c (127.24.31.254:34273), c14d0c97d10e4737acb92a280c143ed1 (127.24.31.253:44765)
I20260809 23:02:01.935457 7890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:02:01.935940 7890 raft_consensus.cc:686] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [term 1 LEADER]: Becoming Leader. State: Replica: a749bca384374a64897f5b80d44460be, State: Running, Role: LEADER
I20260809 23:02:01.936849 7890 consensus_queue.cc:237] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } }
I20260809 23:02:01.951745 7754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a749bca384374a64897f5b80d44460be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14d0c97d10e4737acb92a280c143ed1"
I20260809 23:02:01.952041 7754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:01.957090 7754 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a749bca384374a64897f5b80d44460be in term 1.
I20260809 23:02:01.960650 7943 sys_catalog.cc:455] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [sys.catalog]: SysCatalogTable state changed. Reason: New leader a749bca384374a64897f5b80d44460be. Latest consensus state: current_term: 1 leader_uuid: "a749bca384374a64897f5b80d44460be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:01.961251 7943 sys_catalog.cc:458] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:01.968734 7945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:02:01.971359 7945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:02:01.987583 7754 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [term 1 FOLLOWER]: Refusing update from remote peer a749bca384374a64897f5b80d44460be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:02:01.993592 7943 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [LEADER]: Connected to new peer: Peer: permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:02:02.002072 7667 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [term 1 FOLLOWER]: Refusing update from remote peer a749bca384374a64897f5b80d44460be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:02:02.003369 7943 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [LEADER]: Connected to new peer: Peer: permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:02:02.035295 7908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a749bca384374a64897f5b80d44460be. Latest consensus state: current_term: 1 leader_uuid: "a749bca384374a64897f5b80d44460be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:02.035633 7908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:02:02.041105 7946 mvcc.cc:205] Tried to move back new op lower bound from 7316752474049372160 to 7316752473882087424. Current Snapshot: MvccSnapshot[applied={T|T < 7316752474049372160}]
I20260809 23:02:02.046816 7908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a749bca384374a64897f5b80d44460be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:02.047117 7908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:02:02.054032 7890 sys_catalog.cc:455] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a749bca384374a64897f5b80d44460be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:02.054414 7890 sys_catalog.cc:458] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:02.057860 7945 catalog_manager.cc:1347] Generated new cluster ID: 5f13b92d084443f5a065ac9a940d7853
I20260809 23:02:02.057967 7945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:02:02.074592 7926 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a749bca384374a64897f5b80d44460be. Latest consensus state: current_term: 1 leader_uuid: "a749bca384374a64897f5b80d44460be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:02.074990 7926 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:02:02.078299 7890 sys_catalog.cc:455] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a749bca384374a64897f5b80d44460be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:02.078760 7890 sys_catalog.cc:458] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:02.082410 7947 mvcc.cc:205] Tried to move back new op lower bound from 7316752474049372160 to 7316752473882087424. Current Snapshot: MvccSnapshot[applied={T|T < 7316752474049372160}]
I20260809 23:02:02.095278 7926 sys_catalog.cc:455] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a749bca384374a64897f5b80d44460be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90fe5278f02e4621aabca0cfbb05c67c" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34273 } } peers { permanent_uuid: "c14d0c97d10e4737acb92a280c143ed1" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 44765 } } peers { permanent_uuid: "a749bca384374a64897f5b80d44460be" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 46127 } } }
I20260809 23:02:02.095571 7926 sys_catalog.cc:458] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:02:02.120091 7945 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:02:02.122426 7945 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:02:02.141417 7945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a749bca384374a64897f5b80d44460be: Generated new TSK 0
I20260809 23:02:02.142477 7945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:02:02.625416 7923 catalog_manager.cc:1259] Loaded cluster ID: 5f13b92d084443f5a065ac9a940d7853
I20260809 23:02:02.625535 7923 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c: loading cluster ID for follower catalog manager: success
I20260809 23:02:02.628017 7923 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c: acquiring CA information for follower catalog manager: success
I20260809 23:02:02.629853 7923 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 90fe5278f02e4621aabca0cfbb05c67c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:02:02.721808 7939 catalog_manager.cc:1259] Loaded cluster ID: 5f13b92d084443f5a065ac9a940d7853
I20260809 23:02:02.721918 7939 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1: loading cluster ID for follower catalog manager: success
I20260809 23:02:02.724350 7939 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1: acquiring CA information for follower catalog manager: success
I20260809 23:02:02.726151 7939 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c14d0c97d10e4737acb92a280c143ed1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:02:02.734294 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 7615
I20260809 23:02:02.735503 7689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:02.845533 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 7615
I20260809 23:02:02.867426 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 7703
I20260809 23:02:02.868566 7777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:02.976116 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 7703
I20260809 23:02:02.999651 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 7795
I20260809 23:02:03.000855 7867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:03.131572 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 7795
2026-08-09T23:02:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4610 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:02:03 dist-test-slave-vxj2 krb5kdc[7970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:02:03 dist-test-slave-vxj2 krb5kdc[7970](info): set up 2 sockets
Aug 09 23:02:03 dist-test-slave-vxj2 krb5kdc[7970](info): commencing operation
krb5kdc: starting...
W20260809 23:02:04.331007 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.012s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:02:04 dist-test-slave-vxj2 krb5kdc[7970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:02:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:02:04.592374 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:43319
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:43319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:04.935827 7986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:04.936111 7986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:04.936201 7986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:04.943288 7986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:02:04.943390 7986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:04.943436 7986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:04.943493 7986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:02:04.943537 7986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:02:04.953099 7986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:43319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:43319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:04.954386 7986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:04.955611 7986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:04.967878 7994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:04.967862 7998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:04.967842 7996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:04.968950 7986 server_base.cc:1034] running on GCE node
I20260809 23:02:04.969849 7986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:04.971271 7986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:04.972501 7986 hybrid_clock.cc:648] HybridClock initialized: now 1786316524972464 us; error 49 us; skew 500 ppm
I20260809 23:02:04.972889 7986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:04 dist-test-slave-vxj2 krb5kdc[7970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316524, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:04.976023 7986 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:02:04.977538 7986 webserver.cc:460] Webserver started at http://127.24.31.254:36561/ using document root <none> and password file <none>
I20260809 23:02:04.978220 7986 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:04.978371 7986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:04.978685 7986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:04.981727 7986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "1a02138ed607485f83130cbeb524c8d5"
format_stamp: "Formatted at 2026-08-09 23:02:04 on dist-test-slave-vxj2"
server_key: "8659cf237f7dcd7cf2f08165b42e314c"
server_key_iv: "2f6318bea458fd99e104f2739655425d"
server_key_version: "encryptionkey@0"
I20260809 23:02:04.982506 7986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "1a02138ed607485f83130cbeb524c8d5"
format_stamp: "Formatted at 2026-08-09 23:02:04 on dist-test-slave-vxj2"
server_key: "8659cf237f7dcd7cf2f08165b42e314c"
server_key_iv: "2f6318bea458fd99e104f2739655425d"
server_key_version: "encryptionkey@0"
I20260809 23:02:04.987510 7986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:02:04.990898 8007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:04.992182 7986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:02:04.992370 7986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "1a02138ed607485f83130cbeb524c8d5"
format_stamp: "Formatted at 2026-08-09 23:02:04 on dist-test-slave-vxj2"
server_key: "8659cf237f7dcd7cf2f08165b42e314c"
server_key_iv: "2f6318bea458fd99e104f2739655425d"
server_key_version: "encryptionkey@0"
I20260809 23:02:04.992578 7986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:05.004978 7986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:05.009063 7986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:05.009402 7986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:05.028268 7986 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:43319
I20260809 23:02:05.028309 8066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:43319 every 8 connection(s)
I20260809 23:02:05.029816 7986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:02:05.033736 8067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:05.039212 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 7986
I20260809 23:02:05.039498 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:02:05.040166 24703 external_mini_cluster.cc:1428] Setting key ac73e5095557e756d8daab4f9e041b66
I20260809 23:02:05.045325 8067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5: Bootstrap starting.
I20260809 23:02:05.049551 8067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:05.050897 8067 log.cc:824] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:05.055033 8067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5: No bootstrap required, opened a new log
Aug 09 23:02:05 dist-test-slave-vxj2 krb5kdc[7970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:05.064039 8067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02138ed607485f83130cbeb524c8d5" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43319 } }
I20260809 23:02:05.064538 8067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:05.064677 8067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a02138ed607485f83130cbeb524c8d5, State: Initialized, Role: FOLLOWER
I20260809 23:02:05.065420 8067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02138ed607485f83130cbeb524c8d5" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43319 } }
I20260809 23:02:05.065737 8067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:02:05.065755 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:05.041992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.24.31.254:43319)
0809 23:02:05.042837 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:05.042848 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:05.042893 (+ 45us) server_negotiation.cc:407] Connection header received
0809 23:02:05.044555 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:05.044574 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:05.044935 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:05.045381 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:05.047480 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:05.048815 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:05.051064 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:05.052370 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:05.052589 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:05.057972 (+ 5383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:05.057981 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:05.057995 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:05.058035 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:05.061432 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:05.062183 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:05.062193 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:05.062199 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:05.062377 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:05.062805 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:05.062809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:05.062811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:05.063328 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:05.063623 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:05.064080 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 23:02:05.064545 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":413,"threads_started":1}
I20260809 23:02:05.065936 8067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:02:05.066226 8067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:05.069499 8067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02138ed607485f83130cbeb524c8d5" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43319 } }
I20260809 23:02:05.070147 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:02:05.070104 8067 leader_election.cc:302] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a02138ed607485f83130cbeb524c8d5; no voters:
I20260809 23:02:05.070806 8067 leader_election.cc:288] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:02:05.070953 8072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:02:05.072995 8072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [term 1 LEADER]: Becoming Leader. State: Replica: 1a02138ed607485f83130cbeb524c8d5, State: Running, Role: LEADER
I20260809 23:02:05.073521 8067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:02:05.073772 8072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02138ed607485f83130cbeb524c8d5" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43319 } }
I20260809 23:02:05.077026 8076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a02138ed607485f83130cbeb524c8d5. Latest consensus state: current_term: 1 leader_uuid: "1a02138ed607485f83130cbeb524c8d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02138ed607485f83130cbeb524c8d5" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43319 } } }
I20260809 23:02:05.077419 8076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:05.077420 8075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a02138ed607485f83130cbeb524c8d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a02138ed607485f83130cbeb524c8d5" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 43319 } } }
I20260809 23:02:05.077935 8075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5 [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:05.081825 8084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:02:05.086051 8084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:02:05.094844 8084 catalog_manager.cc:1347] Generated new cluster ID: b69d60e852524465a90d411a8255b542
I20260809 23:02:05.094976 8084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:02:05.106187 8084 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:02:05.107597 8084 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:02:05.114251 8084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a02138ed607485f83130cbeb524c8d5: Generated new TSK 0
I20260809 23:02:05.115043 8084 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:02:05.132790 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 7986
I20260809 23:02:05.134227 8062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:05.225006 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 7986
2026-08-09T23:02:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2087 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:02:05 dist-test-slave-vxj2 krb5kdc[8101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:02:05 dist-test-slave-vxj2 krb5kdc[8101](info): set up 2 sockets
Aug 09 23:02:05 dist-test-slave-vxj2 krb5kdc[8101](info): commencing operation
krb5kdc: starting...
W20260809 23:02:06.411459 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:02:06 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:02:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:02:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:02:06.678459 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39623
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:07.009825 8123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:07.010185 8123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:07.010349 8123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:07.017695 8123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:02:07.017855 8123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:07.017968 8123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:07.018065 8123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:02:07.018150 8123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:02:07.028054 8123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:07.029505 8123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:07.030838 8123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:07.042618 8128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:07.042712 8130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:07.042830 8132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:07.043669 8123 server_base.cc:1034] running on GCE node
I20260809 23:02:07.044521 8123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:07.046038 8123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:07.047298 8123 hybrid_clock.cc:648] HybridClock initialized: now 1786316527047250 us; error 31 us; skew 500 ppm
I20260809 23:02:07.047672 8123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:07 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316527, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:07.054198 8123 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:02:07.055821 8123 webserver.cc:460] Webserver started at http://127.24.31.254:45991/ using document root <none> and password file <none>
I20260809 23:02:07.056584 8123 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:07.056756 8123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:07.057089 8123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:07.060263 8123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "8dee14416d0c45a48be82c9cc5bbea80"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "d0c93994f80da38ad3f8765e6a456c63"
server_key_iv: "597023065a5a75cd211fa03c09b5f60a"
server_key_version: "encryptionkey@0"
I20260809 23:02:07.061112 8123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "8dee14416d0c45a48be82c9cc5bbea80"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "d0c93994f80da38ad3f8765e6a456c63"
server_key_iv: "597023065a5a75cd211fa03c09b5f60a"
server_key_version: "encryptionkey@0"
I20260809 23:02:07.066812 8123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:02:07.070438 8141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:07.071812 8123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:02:07.072010 8123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "8dee14416d0c45a48be82c9cc5bbea80"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "d0c93994f80da38ad3f8765e6a456c63"
server_key_iv: "597023065a5a75cd211fa03c09b5f60a"
server_key_version: "encryptionkey@0"
I20260809 23:02:07.072211 8123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:07.091454 8123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:07.095794 8123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:07.096089 8123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:07.114548 8123 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:39623
I20260809 23:02:07.114658 8201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:39623 every 8 connection(s)
I20260809 23:02:07.115926 8123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:02:07.119839 8202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:07.124625 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 8123
I20260809 23:02:07.124871 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:02:07.125761 24703 external_mini_cluster.cc:1428] Setting key fae313bed22789a0f9d25c74406f4649
I20260809 23:02:07.131948 8202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Bootstrap starting.
I20260809 23:02:07.135689 8202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:07.137058 8202 log.cc:824] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:07.140759 8202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: No bootstrap required, opened a new log
Aug 09 23:02:07 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:07.148633 8202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:07.148988 8202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:07.149118 8202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dee14416d0c45a48be82c9cc5bbea80, State: Initialized, Role: FOLLOWER
I20260809 23:02:07.149840 8202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:07.150096 8202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:02:07.150266 8202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:02:07.150506 8202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:07.151033 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:07.127907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.24.31.254:39623)
0809 23:02:07.128746 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:07.128757 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:07.128797 (+ 40us) server_negotiation.cc:407] Connection header received
0809 23:02:07.130606 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:07.130629 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:07.130990 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:07.131292 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:07.133298 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:07.134641 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:07.136646 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:07.137868 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:07.138063 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:07.143276 (+ 5213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:07.143287 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:07.143300 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:07.143350 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:07.146644 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:07.147392 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:07.147398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:07.147404 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:07.147585 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:07.147991 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:07.147996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:07.147999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:07.148510 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:07.148834 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:07.149286 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 23:02:07.149781 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":382,"threads_started":1}
I20260809 23:02:07.153412 8202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:07.154001 8202 leader_election.cc:302] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dee14416d0c45a48be82c9cc5bbea80; no voters:
I20260809 23:02:07.154558 8202 leader_election.cc:288] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:02:07.154712 8207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:02:07.155537 8207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 1 LEADER]: Becoming Leader. State: Replica: 8dee14416d0c45a48be82c9cc5bbea80, State: Running, Role: LEADER
I20260809 23:02:07.156026 8207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:07.156801 8202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:02:07.159016 8209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dee14416d0c45a48be82c9cc5bbea80. Latest consensus state: current_term: 1 leader_uuid: "8dee14416d0c45a48be82c9cc5bbea80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } } }
I20260809 23:02:07.159327 8209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:07.164688 8208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dee14416d0c45a48be82c9cc5bbea80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } } }
I20260809 23:02:07.164997 8208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: This master's current role is: LEADER
W20260809 23:02:07.169844 8223 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:02:07.169978 8223 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:02:07.170127 8214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:02:07.171926 8214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:02:07.180126 8214 catalog_manager.cc:1347] Generated new cluster ID: 0e4ab2ce44054bb5af41de6cc5207ebf
I20260809 23:02:07.180277 8214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:02:07.214303 8214 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:02:07.216246 8214 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:02:07.230742 8214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Generated new TSK 0
I20260809 23:02:07.231760 8214 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 23:02:07.302534 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39623
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:07.648475 8229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:07.648783 8229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:07.648962 8229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:07.656824 8229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:07.656972 8229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:07.657239 8229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:02:07.668493 8229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:07.669915 8229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:07.671353 8229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:07.685214 8235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:07.685111 8237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:07.685179 8239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:07.686183 8229 server_base.cc:1034] running on GCE node
I20260809 23:02:07.686730 8229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:07.687566 8229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:07.688764 8229 hybrid_clock.cc:648] HybridClock initialized: now 1786316527688723 us; error 30 us; skew 500 ppm
I20260809 23:02:07.689080 8229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:07 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316527, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:07.692224 8229 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:02:07.693636 8229 webserver.cc:460] Webserver started at http://127.24.31.193:38925/ using document root <none> and password file <none>
I20260809 23:02:07.694264 8229 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:07.694365 8229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:07.694602 8229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:07.697475 8229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "d9fdbedf577c4f57acd051353e3383c5"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "3b5cea9003e660ab3466ebb79c9972fd"
server_key_iv: "904a588d54dc1d53ce824ef45ee35afd"
server_key_version: "encryptionkey@0"
I20260809 23:02:07.698205 8229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "d9fdbedf577c4f57acd051353e3383c5"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "3b5cea9003e660ab3466ebb79c9972fd"
server_key_iv: "904a588d54dc1d53ce824ef45ee35afd"
server_key_version: "encryptionkey@0"
I20260809 23:02:07.703225 8229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:02:07.706453 8246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:07.707774 8229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:02:07.707978 8229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "d9fdbedf577c4f57acd051353e3383c5"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "3b5cea9003e660ab3466ebb79c9972fd"
server_key_iv: "904a588d54dc1d53ce824ef45ee35afd"
server_key_version: "encryptionkey@0"
I20260809 23:02:07.708186 8229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:07.724599 8229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:07.728497 8229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:07.728765 8229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:07.729796 8229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:07.731475 8229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:07.731599 8229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:07.731748 8229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:07.731817 8229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:07.764451 8229 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:38061
I20260809 23:02:07.764474 8381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:38061 every 8 connection(s)
I20260809 23:02:07.765805 8229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:02:07.771245 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 8229
I20260809 23:02:07.771462 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:02:07.772378 24703 external_mini_cluster.cc:1428] Setting key 1176c0ba29cc4a811e4cc19db6b358d7
Aug 09 23:02:07 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316527, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:07.788282 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:07.768427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:42043 (local address 127.24.31.254:39623)
0809 23:02:07.768902 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:07.768906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:07.769732 (+ 826us) server_negotiation.cc:407] Connection header received
0809 23:02:07.771958 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:07.771963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:07.772030 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:07.772124 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:07.774094 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:07.775039 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:07.777690 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:07.778101 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:07.778201 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:07.782439 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:07.782445 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:07.782448 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:07.782482 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:07.785065 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:07.785862 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:07.785868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:07.785870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:07.786019 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:07.786495 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:07.786499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:07.786501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:07.786777 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:07.786940 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:07.787663 (+ 723us) server_negotiation.cc:299] Negotiation successful
0809 23:02:07.787912 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":266,"threads_started":1}
I20260809 23:02:07.789456 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:07.768812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:42043)
0809 23:02:07.769569 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:07.769611 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:07.771663 (+ 2052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:07.772446 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:07.772461 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:07.773216 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:07.773919 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:07.773937 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:07.775220 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:07.775227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:07.777546 (+ 2319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:07.777564 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:07.778336 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:07.778345 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:07.778683 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:07.779411 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:07.779454 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:07.782194 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:07.785245 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:07.785255 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:07.785281 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:07.785724 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:07.786139 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:07.786144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:07.786148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:07.786353 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:07.786913 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:07.786920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:07.787276 (+ 356us) client_negotiation.cc:769] Sending connection context
0809 23:02:07.787546 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 23:02:07.788085 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":324,"threads_started":1}
I20260809 23:02:07.791115 8382 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39623
I20260809 23:02:07.791445 8382 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:07.792393 8382 heartbeater.cc:511] Master 127.24.31.254:39623 requested a full tablet report, sending...
I20260809 23:02:07.794659 8161 ts_manager.cc:194] Registered new tserver with Master: d9fdbedf577c4f57acd051353e3383c5 (127.24.31.193:38061)
I20260809 23:02:07.796100 8161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:42043
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 23:02:07.833732 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39623
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:08.164631 8389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:08.164907 8389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:08.165002 8389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:08.172298 8389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:08.172402 8389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:08.172510 8389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:02:08.182027 8389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:08.183256 8389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:08.184507 8389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:08.197568 8394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:08.197659 8395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:08.198357 8397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:08.198925 8389 server_base.cc:1034] running on GCE node
I20260809 23:02:08.199504 8389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:08.200445 8389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:08.201673 8389 hybrid_clock.cc:648] HybridClock initialized: now 1786316528201636 us; error 57 us; skew 500 ppm
I20260809 23:02:08.202000 8389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:08 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316528, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:08.205150 8389 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:02:08.206665 8389 webserver.cc:460] Webserver started at http://127.24.31.194:36411/ using document root <none> and password file <none>
I20260809 23:02:08.207356 8389 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:08.207502 8389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:08.207811 8389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:08.210744 8389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "a3b583f140284b668f7d2faab8b1c954"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "c4b51bf421affe0655d76fe6c6347e37"
server_key_iv: "69a170623aa1eef295eb0300daee95fd"
server_key_version: "encryptionkey@0"
I20260809 23:02:08.211548 8389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "a3b583f140284b668f7d2faab8b1c954"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "c4b51bf421affe0655d76fe6c6347e37"
server_key_iv: "69a170623aa1eef295eb0300daee95fd"
server_key_version: "encryptionkey@0"
I20260809 23:02:08.216729 8389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 23:02:08.220180 8404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:08.221534 8389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:08.221715 8389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "a3b583f140284b668f7d2faab8b1c954"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "c4b51bf421affe0655d76fe6c6347e37"
server_key_iv: "69a170623aa1eef295eb0300daee95fd"
server_key_version: "encryptionkey@0"
I20260809 23:02:08.221918 8389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:08.248900 8389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:08.253481 8389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:08.253803 8389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:08.254791 8389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:08.256323 8389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:08.256433 8389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:08.256594 8389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:08.256680 8389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:08.293413 8389 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:41409
I20260809 23:02:08.293462 8547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:41409 every 8 connection(s)
I20260809 23:02:08.294700 8389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:02:08.299182 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 8389
I20260809 23:02:08.299404 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:02:08.299984 24703 external_mini_cluster.cc:1428] Setting key ee9f31de0b85d42c7ffd45ccec1e541d
Aug 09 23:02:08 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316528, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:08.318930 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.297323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:47367 (local address 127.24.31.254:39623)
0809 23:02:08.297770 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:08.297775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:08.298662 (+ 887us) server_negotiation.cc:407] Connection header received
0809 23:02:08.301147 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:08.301171 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:08.301273 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:08.301409 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:08.303195 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.303926 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.306459 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.306880 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.306992 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.311593 (+ 4601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:08.311598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:08.311602 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:08.311636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:08.315164 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:08.316147 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:08.316156 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:08.316162 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:08.316317 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:08.316759 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:08.316767 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:08.316773 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:08.317217 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:08.317371 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:08.318148 (+ 777us) server_negotiation.cc:299] Negotiation successful
0809 23:02:08.318412 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":239,"threads_started":1}
I20260809 23:02:08.319545 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.297782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:47367)
0809 23:02:08.298506 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:08.298543 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:08.300885 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:08.301637 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:08.301645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:08.302213 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:08.303007 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.303036 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.304111 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.304120 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.306304 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.306328 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.307114 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.307124 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.307469 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.308247 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:08.308296 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:08.311359 (+ 3063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:08.315315 (+ 3956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:08.315322 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:08.315334 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:08.315990 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:08.316435 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:08.316439 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:08.316441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:08.316627 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:08.317348 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:08.317354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:08.317698 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 23:02:08.317967 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 23:02:08.318373 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":306,"threads_started":1}
I20260809 23:02:08.321070 8548 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39623
I20260809 23:02:08.321415 8548 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:08.322177 8548 heartbeater.cc:511] Master 127.24.31.254:39623 requested a full tablet report, sending...
I20260809 23:02:08.323676 8161 ts_manager.cc:194] Registered new tserver with Master: a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409)
I20260809 23:02:08.324553 8161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:47367
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 23:02:08.363965 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39623
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:08.705987 8554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:08.706259 8554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:08.706353 8554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:08.714000 8554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:08.714104 8554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:08.714216 8554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:02:08.724145 8554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:39623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:08.725512 8554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:08.726955 8554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:08.740115 8559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:08.740167 8562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:08.740070 8560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:08.741348 8554 server_base.cc:1034] running on GCE node
I20260809 23:02:08.741955 8554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:08.742854 8554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:08.744077 8554 hybrid_clock.cc:648] HybridClock initialized: now 1786316528744032 us; error 59 us; skew 500 ppm
I20260809 23:02:08.744379 8554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:08 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316528, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:08.747530 8554 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:02:08.748898 8554 webserver.cc:460] Webserver started at http://127.24.31.195:38607/ using document root <none> and password file <none>
I20260809 23:02:08.749579 8554 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:08.749688 8554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:08.749934 8554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:08.752701 8554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "81c56cc956994e43b7b56f91f659c46f"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "83635f83dd31552fcdeae5f9e79ce34f"
server_key_iv: "06a0592e4b3b526cf9d4825b2a36e946"
server_key_version: "encryptionkey@0"
I20260809 23:02:08.753536 8554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "81c56cc956994e43b7b56f91f659c46f"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "83635f83dd31552fcdeae5f9e79ce34f"
server_key_iv: "06a0592e4b3b526cf9d4825b2a36e946"
server_key_version: "encryptionkey@0"
I20260809 23:02:08.758858 8554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:02:08.762223 8569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:08.763576 8554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:08.763763 8554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "81c56cc956994e43b7b56f91f659c46f"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "83635f83dd31552fcdeae5f9e79ce34f"
server_key_iv: "06a0592e4b3b526cf9d4825b2a36e946"
server_key_version: "encryptionkey@0"
I20260809 23:02:08.764004 8554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:08.784307 8554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:08.788630 8554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:08.788947 8554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:08.790016 8554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:08.791394 8554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:08.791501 8554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:08.791663 8554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:08.791740 8554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:08.799885 8382 heartbeater.cc:503] Master 127.24.31.254:39623 was elected leader, sending a full tablet report...
I20260809 23:02:08.827126 8554 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:38789
I20260809 23:02:08.827186 8704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:38789 every 8 connection(s)
I20260809 23:02:08.828418 8554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:02:08.831787 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 8554
I20260809 23:02:08.831992 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:02:08.832659 24703 external_mini_cluster.cc:1428] Setting key a94975a9f71b7f05e7c0cfd3cdb6c965
Aug 09 23:02:08 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316528, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:08.849720 8707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.831087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:54261 (local address 127.24.31.254:39623)
0809 23:02:08.831694 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:08.831702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:08.832805 (+ 1103us) server_negotiation.cc:407] Connection header received
0809 23:02:08.835108 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:08.835112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:08.835189 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:08.835267 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:08.837424 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.838189 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.839780 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.840250 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.840348 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.843668 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:08.843674 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:08.843677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:08.843709 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:08.846414 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:08.847327 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:08.847331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:08.847334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:08.847496 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:08.848060 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:08.848065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:08.848068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:08.848459 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:08.848587 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:08.849272 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 23:02:08.849445 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":318,"threads_started":1}
I20260809 23:02:08.850658 8709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.831692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:54261)
0809 23:02:08.832621 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:08.832685 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:08.834797 (+ 2112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:08.835532 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:08.835542 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:08.836631 (+ 1089us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:08.837270 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.837290 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.838318 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.838322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.839659 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.839671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.840485 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.840493 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.840709 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.841372 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:08.841399 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:08.843485 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:08.846570 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:08.846578 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:08.846593 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:08.847187 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:08.847653 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:08.847657 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:08.847659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:08.847923 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:08.848595 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:08.848601 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:08.848889 (+ 288us) client_negotiation.cc:769] Sending connection context
0809 23:02:08.849110 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 23:02:08.849556 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":356,"threads_started":1}
I20260809 23:02:08.852049 8705 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39623
I20260809 23:02:08.852340 8705 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:08.853065 8705 heartbeater.cc:511] Master 127.24.31.254:39623 requested a full tablet report, sending...
I20260809 23:02:08.854465 8161 ts_manager.cc:194] Registered new tserver with Master: 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789)
I20260809 23:02:08.855273 8161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:54261
I20260809 23:02:08.860577 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:02:08.878328 8707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.865262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54562 (local address 127.24.31.254:39623)
0809 23:02:08.865454 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:08.865461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:08.865880 (+ 419us) server_negotiation.cc:407] Connection header received
0809 23:02:08.866092 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:08.866099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:08.866179 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:08.866283 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:08.867888 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.868706 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.870171 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.870570 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.870704 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.873135 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:08.873140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:08.873144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:08.873224 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:08.875690 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:08.876358 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:08.876365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:08.876368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:08.876505 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:08.876979 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:08.876982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:08.876984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:08.877362 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:08.877563 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:08.877841 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 23:02:08.878094 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 23:02:08.882288 8161 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54562:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:02:08.889218 8161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:02:08.922631 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.901566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50596 (local address 127.24.31.194:41409)
0809 23:02:08.913329 (+ 11763us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:08.913343 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:08.913381 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:02:08.913499 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:08.913509 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:08.913790 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:08.913991 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:08.915471 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.917055 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.919800 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.921229 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.921380 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.921591 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:08.922079 (+ 488us) server_negotiation.cc:299] Negotiation successful
0809 23:02:08.922253 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11499,"thread_start_us":11365,"threads_started":1}
I20260809 23:02:08.922688 8720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.901334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:41409 (local address 127.0.0.1:50596)
0809 23:02:08.912532 (+ 11198us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:08.912552 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:08.912682 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:08.914613 (+ 1931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:08.914620 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:08.914654 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:08.915251 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.915265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.917245 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.917251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.919630 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.919644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.921713 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.921717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.921840 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.921871 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 23:02:08.922033 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 23:02:08.922237 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11073,"mutex_wait_us":187,"thread_start_us":10922,"threads_started":1}
I20260809 23:02:08.923615 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.901541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60102 (local address 127.24.31.193:38061)
0809 23:02:08.914817 (+ 13276us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:08.914827 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:08.914858 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:02:08.914953 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:08.914961 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:08.915195 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:08.915452 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:08.916056 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.917535 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.920489 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.922315 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.922462 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.922687 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:08.923198 (+ 511us) server_negotiation.cc:299] Negotiation successful
0809 23:02:08.923337 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13092,"thread_start_us":12851,"threads_started":1}
I20260809 23:02:08.923660 8725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.901159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60102)
0809 23:02:08.911150 (+ 9991us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:08.911210 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:08.912518 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:08.915429 (+ 2911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:08.915436 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:08.915464 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:08.915856 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.915873 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.917782 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.917798 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.920295 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.920310 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.922836 (+ 2526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.922841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.922997 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.923031 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 23:02:08.923133 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 23:02:08.923263 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9582,"mutex_wait_us":44,"thread_start_us":12163,"threads_started":1}
I20260809 23:02:08.924573 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.904204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:38789 (local address 127.0.0.1:38942)
0809 23:02:08.913731 (+ 9527us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:08.913755 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:08.913937 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:08.914678 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:08.914683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:08.914707 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:08.915188 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.915202 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.916967 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.916972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.920298 (+ 3326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.920310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.922437 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.922444 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.922636 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.922682 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 23:02:08.924061 (+ 1379us) client_negotiation.cc:240] Negotiation successful
0809 23:02:08.924193 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9349,"mutex_wait_us":195,"thread_start_us":6375,"threads_started":1}
I20260809 23:02:08.924634 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.905017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.24.31.195:38789)
0809 23:02:08.910708 (+ 5691us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:08.910718 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:08.914104 (+ 3386us) server_negotiation.cc:407] Connection header received
0809 23:02:08.914231 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:08.914241 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:08.914484 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:08.914670 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:08.915380 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.916778 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.920485 (+ 3707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.921901 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.922046 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.922267 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:08.924108 (+ 1841us) server_negotiation.cc:299] Negotiation successful
0809 23:02:08.924272 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5505,"thread_start_us":5325,"threads_started":1}
I20260809 23:02:08.927695 8307 tablet_service.cc:1467] Processing CreateTablet for tablet a1738444458f493cba200d6e06da98d5 (DEFAULT_TABLE table=test-table [id=fe1bd88337404afeb6a68b7da9173d44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:08.928050 8627 tablet_service.cc:1467] Processing CreateTablet for tablet a1738444458f493cba200d6e06da98d5 (DEFAULT_TABLE table=test-table [id=fe1bd88337404afeb6a68b7da9173d44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:08.929009 8307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1738444458f493cba200d6e06da98d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:08.929309 8627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1738444458f493cba200d6e06da98d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:08.937786 8466 tablet_service.cc:1467] Processing CreateTablet for tablet a1738444458f493cba200d6e06da98d5 (DEFAULT_TABLE table=test-table [id=fe1bd88337404afeb6a68b7da9173d44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:08.939195 8466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1738444458f493cba200d6e06da98d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:08.940426 8727 tablet_bootstrap.cc:492] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: Bootstrap starting.
I20260809 23:02:08.942457 8729 tablet_bootstrap.cc:492] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: Bootstrap starting.
I20260809 23:02:08.944614 8727 tablet_bootstrap.cc:654] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:08.945987 8729 tablet_bootstrap.cc:654] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:08.945981 8727 log.cc:824] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:08.947160 8729 log.cc:824] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:08.949190 8727 tablet_bootstrap.cc:492] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: No bootstrap required, opened a new log
I20260809 23:02:08.949505 8727 ts_tablet_manager.cc:1397] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260809 23:02:08.950242 8729 tablet_bootstrap.cc:492] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: No bootstrap required, opened a new log
I20260809 23:02:08.950505 8729 ts_tablet_manager.cc:1397] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 23:02:08.953123 8732 tablet_bootstrap.cc:492] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: Bootstrap starting.
I20260809 23:02:08.955740 8732 tablet_bootstrap.cc:654] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:08.956982 8732 log.cc:824] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:08.959640 8732 tablet_bootstrap.cc:492] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: No bootstrap required, opened a new log
I20260809 23:02:08.959900 8732 ts_tablet_manager.cc:1397] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 23:02:08.960412 8727 raft_consensus.cc:348] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:08.960884 8727 raft_consensus.cc:374] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:08.961050 8727 raft_consensus.cc:729] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9fdbedf577c4f57acd051353e3383c5, State: Initialized, Role: FOLLOWER
I20260809 23:02:08.960968 8729 raft_consensus.cc:348] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:08.961486 8729 raft_consensus.cc:374] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:08.961597 8729 raft_consensus.cc:729] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c56cc956994e43b7b56f91f659c46f, State: Initialized, Role: FOLLOWER
I20260809 23:02:08.961740 8727 consensus_queue.cc:260] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:08.962183 8729 consensus_queue.cc:260] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:08.965564 8727 ts_tablet_manager.cc:1428] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260809 23:02:08.966660 8705 heartbeater.cc:503] Master 127.24.31.254:39623 was elected leader, sending a full tablet report...
I20260809 23:02:08.968245 8729 ts_tablet_manager.cc:1428] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
I20260809 23:02:08.970947 8732 raft_consensus.cc:348] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:08.971432 8732 raft_consensus.cc:374] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:08.971566 8732 raft_consensus.cc:729] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3b583f140284b668f7d2faab8b1c954, State: Initialized, Role: FOLLOWER
I20260809 23:02:08.972092 8732 consensus_queue.cc:260] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:08.973079 8548 heartbeater.cc:503] Master 127.24.31.254:39623 was elected leader, sending a full tablet report...
I20260809 23:02:08.973559 8732 ts_tablet_manager.cc:1428] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 23:02:08.985517 8737 raft_consensus.cc:482] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:08.985817 8737 raft_consensus.cc:504] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:08.987560 8737 leader_election.cc:288] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9fdbedf577c4f57acd051353e3383c5 (127.24.31.193:38061), 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789)
I20260809 23:02:08.994959 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.988137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:45035 (local address 127.24.31.195:38789)
0809 23:02:08.988389 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:08.988395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:08.988895 (+ 500us) server_negotiation.cc:407] Connection header received
0809 23:02:08.989140 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:08.989144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:08.989259 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:08.989356 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:08.990048 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.990802 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.993205 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.993996 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.994193 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.994273 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:08.994569 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 23:02:08.994698 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 23:02:08.995249 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.987846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:52243 (local address 127.24.31.193:38061)
0809 23:02:08.988088 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:08.988097 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:08.988972 (+ 875us) server_negotiation.cc:407] Connection header received
0809 23:02:08.989234 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:08.989239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:08.989306 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:08.989422 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:08.990199 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.990959 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.993348 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.994169 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.994398 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.994482 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:08.994817 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 23:02:08.994954 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 23:02:08.995276 8738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.987835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:38789 (local address 127.24.31.194:45035)
0809 23:02:08.988761 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:08.988783 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:08.988977 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:08.989373 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:08.989378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:08.989411 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:08.989883 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.989892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.990934 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.990948 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.992948 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.992965 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.994223 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.994229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.994392 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.994419 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 23:02:08.994540 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 23:02:08.994673 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":66,"spinlock_wait_cycles":17152,"thread_start_us":499,"threads_started":1}
I20260809 23:02:08.995543 8739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:08.987870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.24.31.194:52243)
0809 23:02:08.988877 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:08.988894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:08.989009 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:08.989436 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:08.989441 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:08.989473 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:08.990002 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.990013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.991127 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.991133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.993126 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:08.993145 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:08.994426 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:08.994434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:08.994630 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:08.994656 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 23:02:08.994787 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 23:02:08.994928 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":5,"thread_start_us":707,"threads_started":1}
I20260809 23:02:08.996570 8331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1738444458f493cba200d6e06da98d5" candidate_uuid: "a3b583f140284b668f7d2faab8b1c954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9fdbedf577c4f57acd051353e3383c5" is_pre_election: true
I20260809 23:02:08.996556 8651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1738444458f493cba200d6e06da98d5" candidate_uuid: "a3b583f140284b668f7d2faab8b1c954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c56cc956994e43b7b56f91f659c46f" is_pre_election: true
I20260809 23:02:08.997277 8331 raft_consensus.cc:2454] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3b583f140284b668f7d2faab8b1c954 in term 0.
I20260809 23:02:08.997277 8651 raft_consensus.cc:2454] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3b583f140284b668f7d2faab8b1c954 in term 0.
I20260809 23:02:08.998018 8405 leader_election.cc:302] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3b583f140284b668f7d2faab8b1c954, d9fdbedf577c4f57acd051353e3383c5; no voters:
I20260809 23:02:08.998391 8737 raft_consensus.cc:2781] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:02:08.998571 8737 raft_consensus.cc:482] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:02:08.998694 8737 raft_consensus.cc:3037] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:09.000909 8737 raft_consensus.cc:504] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:09.001644 8737 leader_election.cc:288] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [CANDIDATE]: Term 1 election: Requested vote from peers d9fdbedf577c4f57acd051353e3383c5 (127.24.31.193:38061), 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789)
I20260809 23:02:09.002157 8331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1738444458f493cba200d6e06da98d5" candidate_uuid: "a3b583f140284b668f7d2faab8b1c954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9fdbedf577c4f57acd051353e3383c5"
I20260809 23:02:09.002274 8651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1738444458f493cba200d6e06da98d5" candidate_uuid: "a3b583f140284b668f7d2faab8b1c954" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c56cc956994e43b7b56f91f659c46f"
I20260809 23:02:09.002378 8331 raft_consensus.cc:3037] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:09.002519 8651 raft_consensus.cc:3037] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:09.004650 8331 raft_consensus.cc:2454] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3b583f140284b668f7d2faab8b1c954 in term 1.
I20260809 23:02:09.004719 8651 raft_consensus.cc:2454] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3b583f140284b668f7d2faab8b1c954 in term 1.
I20260809 23:02:09.005108 8405 leader_election.cc:302] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3b583f140284b668f7d2faab8b1c954, d9fdbedf577c4f57acd051353e3383c5; no voters:
I20260809 23:02:09.005456 8737 raft_consensus.cc:2781] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:02:09.005944 8737 raft_consensus.cc:686] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 1 LEADER]: Becoming Leader. State: Replica: a3b583f140284b668f7d2faab8b1c954, State: Running, Role: LEADER
I20260809 23:02:09.006496 8737 consensus_queue.cc:237] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:09.011173 8161 catalog_manager.cc:5495] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3b583f140284b668f7d2faab8b1c954 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "a3b583f140284b668f7d2faab8b1c954" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } health_report { overall_health: UNKNOWN } } }
W20260809 23:02:09.018105 8383 tablet.cc:2369] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:02:09.046768 8549 tablet.cc:2369] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:02:09.049744 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.043931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.24.31.194:41409)
0809 23:02:09.044163 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:09.044169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:09.044287 (+ 118us) server_negotiation.cc:407] Connection header received
0809 23:02:09.044557 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:09.044561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:09.044637 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:09.044812 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:02:09.045226 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:09.045952 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:09.047502 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:09.047921 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:09.048062 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:09.048589 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:02:09.049075 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:02:09.049258 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:09.049379 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 23:02:09.049470 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 23:02:09.058254 8331 raft_consensus.cc:1261] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Refusing update from remote peer a3b583f140284b668f7d2faab8b1c954: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:02:09.058243 8651 raft_consensus.cc:1261] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [term 1 FOLLOWER]: Refusing update from remote peer a3b583f140284b668f7d2faab8b1c954: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:02:09.059417 8740 consensus_queue.cc:1035] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:02:09.059796 8737 consensus_queue.cc:1035] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:02:09.073541 8744 mvcc.cc:205] Tried to move back new op lower bound from 7316752503010865152 to 7316752502815019008. Current Snapshot: MvccSnapshot[applied={T|T < 7316752503010865152}]
I20260809 23:02:09.087582 8160 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:02:09.087940 8160 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54562:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:02:09.091902 8160 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Sending DeleteTablet for 3 replicas of tablet a1738444458f493cba200d6e06da98d5
I20260809 23:02:09.093161 8307 tablet_service.cc:1514] Processing DeleteTablet for tablet a1738444458f493cba200d6e06da98d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:09 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:60102
I20260809 23:02:09.093261 8627 tablet_service.cc:1514] Processing DeleteTablet for tablet a1738444458f493cba200d6e06da98d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:09 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:38942
I20260809 23:02:09.093264 8466 tablet_service.cc:1514] Processing DeleteTablet for tablet a1738444458f493cba200d6e06da98d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:09 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:50596
I20260809 23:02:09.094405 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 8229
I20260809 23:02:09.104797 8756 tablet_replica.cc:330] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: stopping tablet replica
I20260809 23:02:09.106199 8756 raft_consensus.cc:2229] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:02:09.107555 8756 raft_consensus.cc:2258] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:09.109422 8757 tablet_replica.cc:330] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: stopping tablet replica
I20260809 23:02:09.110021 8757 raft_consensus.cc:2229] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:02:09.110963 8757 raft_consensus.cc:2258] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:09.111191 8756 ts_tablet_manager.cc:1905] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:09.112573 8754 tablet_replica.cc:330] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: stopping tablet replica
I20260809 23:02:09.117900 8757 ts_tablet_manager.cc:1905] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 23:02:09 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 23:02:09.112989 8754 raft_consensus.cc:2229] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:02:09.125267 8754 raft_consensus.cc:2258] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:09.130851 8756 ts_tablet_manager.cc:1918] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:09.133273 8754 ts_tablet_manager.cc:1905] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:09.136744 8757 ts_tablet_manager.cc:1918] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:09.136960 8757 log.cc:1196] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/a1738444458f493cba200d6e06da98d5
I20260809 23:02:09.133301 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.096044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.24.31.193:38061)
0809 23:02:09.096279 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:09.096289 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:09.105997 (+ 9708us) server_negotiation.cc:407] Connection header received
0809 23:02:09.106588 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:09.106600 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:09.106719 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:09.106835 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:09.112200 (+ 5365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:09.113453 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:09.116011 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:09.116705 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:09.116845 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:09.123895 (+ 7050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:09.123913 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:09.123935 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:09.123993 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:09.129055 (+ 5062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:09.130165 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:09.130175 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:09.130180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:09.130370 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:09.130928 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:09.130933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:09.130937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:09.131448 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:09.131823 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:09.132566 (+ 743us) server_negotiation.cc:299] Negotiation successful
0809 23:02:09.132852 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 23:02:09.137867 8756 log.cc:1196] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/a1738444458f493cba200d6e06da98d5
I20260809 23:02:09.138055 8377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:09.138993 8757 ts_tablet_manager.cc:1939] T a1738444458f493cba200d6e06da98d5 P a3b583f140284b668f7d2faab8b1c954: Deleting consensus metadata
I20260809 23:02:09.141078 8143 catalog_manager.cc:4841] TS a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409): tablet a1738444458f493cba200d6e06da98d5 (table test-table [id=fe1bd88337404afeb6a68b7da9173d44]) successfully deleted
I20260809 23:02:09.141992 8754 ts_tablet_manager.cc:1918] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:09.142171 8754 log.cc:1196] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/a1738444458f493cba200d6e06da98d5
I20260809 23:02:09.142794 8754 ts_tablet_manager.cc:1939] T a1738444458f493cba200d6e06da98d5 P 81c56cc956994e43b7b56f91f659c46f: Deleting consensus metadata
I20260809 23:02:09.145419 8147 catalog_manager.cc:4841] TS 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789): tablet a1738444458f493cba200d6e06da98d5 (table test-table [id=fe1bd88337404afeb6a68b7da9173d44]) successfully deleted
I20260809 23:02:09.307355 8756 ts_tablet_manager.cc:1939] T a1738444458f493cba200d6e06da98d5 P d9fdbedf577c4f57acd051353e3383c5: Deleting consensus metadata
I20260809 23:02:09.308050 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 8229
I20260809 23:02:09.336670 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 8389
W20260809 23:02:09.336812 8142 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.31.193:38061 (error 108)
I20260809 23:02:09.339263 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.338678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60116)
0809 23:02:09.338939 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:09.339070 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260809 23:02:09.339535 8142 catalog_manager.cc:4584] TS d9fdbedf577c4f57acd051353e3383c5 (127.24.31.193:38061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1738444458f493cba200d6e06da98d5: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Aug 09 23:02:09 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 23:02:09.355317 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.338188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.24.31.194:41409)
0809 23:02:09.338395 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:09.338401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:09.338418 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:02:09.338477 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:09.338481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:09.338554 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:09.338675 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:09.340356 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:09.341670 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:09.342941 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:09.343614 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:09.343740 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:09.348013 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:09.348031 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:09.348050 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:09.348095 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:09.352127 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:09.352755 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:09.352766 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:09.352770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:09.352941 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:09.353685 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:09.353691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:09.353694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:09.354188 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:09.354537 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:09.354689 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 23:02:09.354924 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 23:02:09.356361 8543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:09.393497 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.392944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60132)
0809 23:02:09.393215 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:09.393328 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 23:02:09.394629 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.394262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60148)
0809 23:02:09.394414 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:09.394501 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 23:02:09.455197 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.454667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60154)
0809 23:02:09.454953 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:09.455047 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260809 23:02:09.456301 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.455931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60160)
0809 23:02:09.456113 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:09.456179 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 23:02:09.481086 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 8389
I20260809 23:02:09.501312 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 8554
Aug 09 23:02:09 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 23:02:09.517357 8763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.502067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38950 (local address 127.24.31.195:38789)
0809 23:02:09.502540 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:09.502545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:09.502561 (+ 16us) server_negotiation.cc:407] Connection header received
0809 23:02:09.502618 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:09.502621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:09.502680 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:09.502784 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:09.504191 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:09.504900 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:09.506295 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:09.506693 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:09.506800 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:09.511430 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:09.511445 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:09.511463 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:09.511502 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:09.514495 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:09.515221 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:09.515229 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:09.515231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:09.515378 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:09.515735 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:09.515739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:09.515741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:09.516137 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:09.516362 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:09.516605 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 23:02:09.516816 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":267,"threads_started":1}
I20260809 23:02:09.518131 8700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:09.537569 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.537023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60174)
0809 23:02:09.537299 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:09.537392 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 23:02:09.538693 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.538314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60188)
0809 23:02:09.538504 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:09.538574 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260809 23:02:09.612973 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 8554
I20260809 23:02:09.632902 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 8123
I20260809 23:02:09.634203 8197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:09.751755 8719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:09.751226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60204)
0809 23:02:09.751442 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:09.751524 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 23:02:09.751788 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 8123
I20260809 23:02:09.775350 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39623
--webserver_interface=127.24.31.254
--webserver_port=45991
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:10.109298 8767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:10.109599 8767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:10.109814 8767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:10.117167 8767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:02:10.117286 8767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:10.117405 8767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:10.117520 8767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:02:10.117602 8767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:02:10.127933 8767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=45991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:10.129371 8767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:10.130750 8767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:10.158607 8776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:10.158634 8779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:10.158676 8777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:10.159621 8767 server_base.cc:1034] running on GCE node
I20260809 23:02:10.160517 8767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:10.162272 8767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:10.163552 8767 hybrid_clock.cc:648] HybridClock initialized: now 1786316530163495 us; error 44 us; skew 500 ppm
I20260809 23:02:10.163962 8767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:10 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:10.167807 8767 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:02:10.169685 8767 webserver.cc:460] Webserver started at http://127.24.31.254:45991/ using document root <none> and password file <none>
I20260809 23:02:10.170463 8767 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:10.170632 8767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:10.176851 8767 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 23:02:10.180095 8786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:10.181520 8767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 23:02:10.181720 8767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "8dee14416d0c45a48be82c9cc5bbea80"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "d0c93994f80da38ad3f8765e6a456c63"
server_key_iv: "597023065a5a75cd211fa03c09b5f60a"
server_key_version: "encryptionkey@0"
I20260809 23:02:10.182840 8767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:10.196934 8767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:10.201099 8767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:10.201406 8767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:10.221493 8767 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:39623
I20260809 23:02:10.221522 8852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:39623 every 8 connection(s)
I20260809 23:02:10.223212 8767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:02:10.230973 8853 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:02:10.231591 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 8767
I20260809 23:02:10.232630 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:38061
--local_ip_for_outbound_sockets=127.24.31.193
--tserver_master_addrs=127.24.31.254:39623
--webserver_port=38925
--webserver_interface=127.24.31.193
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:02:10.236335 8853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Bootstrap starting.
I20260809 23:02:10.258683 8853 log.cc:824] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:10.271668 8853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:02:10.272231 8853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Bootstrap complete.
I20260809 23:02:10.281800 8853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:10.282647 8853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dee14416d0c45a48be82c9cc5bbea80, State: Initialized, Role: FOLLOWER
I20260809 23:02:10.283391 8853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:10.283658 8853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:02:10.283774 8853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:02:10.283983 8853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:02:10.288308 8853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:10.288861 8853 leader_election.cc:302] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dee14416d0c45a48be82c9cc5bbea80; no voters:
I20260809 23:02:10.289588 8853 leader_election.cc:288] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:02:10.289738 8857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:02:10.291182 8857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 2 LEADER]: Becoming Leader. State: Replica: 8dee14416d0c45a48be82c9cc5bbea80, State: Running, Role: LEADER
I20260809 23:02:10.291857 8857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:10.292131 8853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:02:10.294476 8860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dee14416d0c45a48be82c9cc5bbea80. Latest consensus state: current_term: 2 leader_uuid: "8dee14416d0c45a48be82c9cc5bbea80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } } }
I20260809 23:02:10.294895 8860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:10.295547 8858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8dee14416d0c45a48be82c9cc5bbea80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } } }
I20260809 23:02:10.295877 8858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:10.295857 8864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:02:10.312394 8864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:02:10.314052 8864 catalog_manager.cc:1259] Loaded cluster ID: 0e4ab2ce44054bb5af41de6cc5207ebf
I20260809 23:02:10.314227 8864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:02:10.317780 8864 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 23:02:10.336172 8864 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 23:02:10.354514 8864 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:02:10.356446 8864 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Loaded TSK: 0
I20260809 23:02:10.358275 8864 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:02:10.607663 8855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:10.607933 8855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:10.608076 8855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:10.615566 8855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:10.615686 8855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:10.615869 8855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:02:10.627772 8855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:38061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=38925
--tserver_master_addrs=127.24.31.254:39623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:10.629124 8855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:10.630551 8855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:10.645259 8885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:10.645689 8884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:10.645713 8887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:10.647185 8855 server_base.cc:1034] running on GCE node
I20260809 23:02:10.647802 8855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:10.648773 8855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:10.650008 8855 hybrid_clock.cc:648] HybridClock initialized: now 1786316530649950 us; error 55 us; skew 500 ppm
I20260809 23:02:10.650331 8855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:10 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:10.653599 8855 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:02:10.655135 8855 webserver.cc:460] Webserver started at http://127.24.31.193:38925/ using document root <none> and password file <none>
I20260809 23:02:10.655831 8855 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:10.655970 8855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:10.661659 8855 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 23:02:10.664768 8894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:10.666240 8855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:10.666421 8855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "d9fdbedf577c4f57acd051353e3383c5"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "3b5cea9003e660ab3466ebb79c9972fd"
server_key_iv: "904a588d54dc1d53ce824ef45ee35afd"
server_key_version: "encryptionkey@0"
I20260809 23:02:10.667421 8855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:10.692693 8855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:10.697230 8855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:10.697525 8855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:10.698590 8855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:10.700098 8855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:10.700227 8855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:10.700368 8855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:10.700460 8855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:10.735572 8855 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:38061
I20260809 23:02:10.735661 9029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:38061 every 8 connection(s)
I20260809 23:02:10.737103 8855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:02:10.744923 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 8855
I20260809 23:02:10.745757 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:41409
--local_ip_for_outbound_sockets=127.24.31.194
--tserver_master_addrs=127.24.31.254:39623
--webserver_port=36411
--webserver_interface=127.24.31.194
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:02:10 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316530, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:10.763274 9033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:10.740496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:51777)
0809 23:02:10.741410 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:10.741454 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:10.743091 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:10.744451 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:10.744463 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:10.745233 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:10.746065 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:10.746093 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:10.747759 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:10.747767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:10.749891 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:10.749909 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:10.750986 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:10.750995 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:10.751339 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:10.752114 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:10.752148 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:10.754887 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:10.758359 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:10.758370 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:10.758386 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:10.758825 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:10.759278 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:10.759283 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:10.759286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:10.759480 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:10.760574 (+ 1094us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:10.760584 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:10.761042 (+ 458us) client_negotiation.cc:769] Sending connection context
0809 23:02:10.761366 (+ 324us) client_negotiation.cc:240] Negotiation successful
0809 23:02:10.761939 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":390,"threads_started":1}
I20260809 23:02:10.763288 9032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:10.740262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:51777 (local address 127.24.31.254:39623)
0809 23:02:10.740834 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:10.740845 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:10.741602 (+ 757us) server_negotiation.cc:407] Connection header received
0809 23:02:10.743593 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:10.743606 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:10.743913 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:10.744266 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:10.746318 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:10.747575 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:10.750064 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:10.750732 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:10.750889 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:10.755175 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:10.755183 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:10.755196 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:10.755237 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:10.758205 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:10.758979 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:10.758985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:10.758991 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:10.759149 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:10.759614 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:10.759618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:10.759620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:10.760250 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:10.760492 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:10.761601 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0809 23:02:10.762020 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":250,"threads_started":1}
I20260809 23:02:10.766744 9030 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39623
I20260809 23:02:10.767057 9030 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:10.767784 9030 heartbeater.cc:511] Master 127.24.31.254:39623 requested a full tablet report, sending...
I20260809 23:02:10.770399 8805 ts_manager.cc:194] Registered new tserver with Master: d9fdbedf577c4f57acd051353e3383c5 (127.24.31.193:38061)
I20260809 23:02:10.772787 8805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:51777
W20260809 23:02:11.080164 9034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:11.080430 9034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:11.080525 9034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:11.087865 9034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:11.087972 9034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:11.088080 9034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:02:11.097802 9034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:41409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=36411
--tserver_master_addrs=127.24.31.254:39623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:11.099037 9034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:11.100338 9034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:11.113898 9040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:11.113897 9041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:11.114005 9043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:11.115168 9034 server_base.cc:1034] running on GCE node
I20260809 23:02:11.115796 9034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:11.116788 9034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:11.118057 9034 hybrid_clock.cc:648] HybridClock initialized: now 1786316531117960 us; error 80 us; skew 500 ppm
I20260809 23:02:11.118427 9034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:11 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:11.121721 9034 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:02:11.123135 9034 webserver.cc:460] Webserver started at http://127.24.31.194:36411/ using document root <none> and password file <none>
I20260809 23:02:11.123844 9034 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:11.124006 9034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:11.129412 9034 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 23:02:11.132112 9050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:11.133407 9034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:11.133594 9034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "a3b583f140284b668f7d2faab8b1c954"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "c4b51bf421affe0655d76fe6c6347e37"
server_key_iv: "69a170623aa1eef295eb0300daee95fd"
server_key_version: "encryptionkey@0"
I20260809 23:02:11.134539 9034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:11.161067 9034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:11.165798 9034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:11.166097 9034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:11.167169 9034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:11.168540 9034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:11.168696 9034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:11.168819 9034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:11.168906 9034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:11.203480 9034 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:41409
I20260809 23:02:11.203573 9186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:41409 every 8 connection(s)
I20260809 23:02:11.205008 9034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:02:11.205538 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 9034
I20260809 23:02:11.206347 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:38789
--local_ip_for_outbound_sockets=127.24.31.195
--tserver_master_addrs=127.24.31.254:39623
--webserver_port=38607
--webserver_interface=127.24.31.195
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:02:11 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:11.230415 9032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.207980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:37581 (local address 127.24.31.254:39623)
0809 23:02:11.208223 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.208231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.209579 (+ 1348us) server_negotiation.cc:407] Connection header received
0809 23:02:11.212483 (+ 2904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.212488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.212558 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.212645 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:11.214789 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.215502 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.217667 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.218108 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.218258 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.222229 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:11.222239 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:11.222247 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:11.222296 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:11.226370 (+ 4074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:11.227342 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:11.227351 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:11.227357 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:11.227525 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:11.228011 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:11.228018 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:11.228025 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:11.228493 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:11.228669 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.229615 (+ 946us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.229912 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 23:02:11.231310 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.208463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:37581)
0809 23:02:11.209367 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:11.209417 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:11.212167 (+ 2750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:11.212917 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:11.212932 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:11.213774 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:11.214590 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.214617 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.215679 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.215687 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.217490 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.217503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.218352 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.218360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.218588 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.219287 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:11.219310 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:11.221974 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:11.226534 (+ 4560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:11.226543 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:11.226563 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:11.227188 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:11.227640 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:11.227645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:11.227649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:11.227870 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:11.228635 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:11.228644 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:11.229065 (+ 421us) client_negotiation.cc:769] Sending connection context
0809 23:02:11.229410 (+ 345us) client_negotiation.cc:240] Negotiation successful
0809 23:02:11.229958 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":306,"threads_started":1}
I20260809 23:02:11.232877 9187 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39623
I20260809 23:02:11.233287 9187 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:11.234287 9187 heartbeater.cc:511] Master 127.24.31.254:39623 requested a full tablet report, sending...
I20260809 23:02:11.236142 8805 ts_manager.cc:194] Registered new tserver with Master: a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409)
I20260809 23:02:11.237318 8805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:37581
W20260809 23:02:11.549721 9191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:11.550012 9191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:11.550170 9191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:11.557564 9191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:11.557677 9191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:11.557852 9191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:02:11.568091 9191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:38789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=38607
--tserver_master_addrs=127.24.31.254:39623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:11.569393 9191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:11.570724 9191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:11.584093 9196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:11.584125 9197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:11.584414 9199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:11.585074 9191 server_base.cc:1034] running on GCE node
I20260809 23:02:11.585726 9191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:11.586764 9191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:11.587998 9191 hybrid_clock.cc:648] HybridClock initialized: now 1786316531587957 us; error 35 us; skew 500 ppm
I20260809 23:02:11.588325 9191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:11 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:11.591549 9191 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:02:11.593235 9191 webserver.cc:460] Webserver started at http://127.24.31.195:38607/ using document root <none> and password file <none>
I20260809 23:02:11.593938 9191 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:11.594115 9191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:11.599591 9191 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 23:02:11.602466 9207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:11.603755 9191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:11.603945 9191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "81c56cc956994e43b7b56f91f659c46f"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "83635f83dd31552fcdeae5f9e79ce34f"
server_key_iv: "06a0592e4b3b526cf9d4825b2a36e946"
server_key_version: "encryptionkey@0"
I20260809 23:02:11.604966 9191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:11.625814 9191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:11.629812 9191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:11.630112 9191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:11.631115 9191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:11.632467 9191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:11.632584 9191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:11.632719 9191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:11.632815 9191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:11.667959 9191 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:38789
I20260809 23:02:11.668051 9334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:38789 every 8 connection(s)
I20260809 23:02:11.669476 9191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:02:11.679301 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 9191
Aug 09 23:02:11 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316531, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:11.691398 9032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.672210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:48249 (local address 127.24.31.254:39623)
0809 23:02:11.672426 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.672431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.673621 (+ 1190us) server_negotiation.cc:407] Connection header received
0809 23:02:11.675478 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.675483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.675562 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.675653 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:11.677211 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.677945 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.680002 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.680545 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.680702 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.684735 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:11.684740 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:11.684743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:11.684774 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:11.688096 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:11.688820 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:11.688826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:11.688830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:11.688964 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:11.689540 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:11.689546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:11.689549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:11.689825 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:11.690000 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.690709 (+ 709us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.691024 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 23:02:11.692430 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.672667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:48249)
0809 23:02:11.673448 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:11.673493 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:11.675214 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:11.675914 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:11.675925 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:11.676451 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:11.677002 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.677018 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.678098 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.678102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.679827 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.679844 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.681118 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.681128 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.681521 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.682205 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:11.682232 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:11.684518 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:11.688237 (+ 3719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:11.688247 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:11.688263 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:11.688691 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:11.689076 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:11.689080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:11.689084 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:11.689430 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:11.689989 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:11.689995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:11.690325 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 23:02:11.690599 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 23:02:11.691146 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":286,"threads_started":1}
I20260809 23:02:11.694155 9335 heartbeater.cc:347] Connected to a master server at 127.24.31.254:39623
I20260809 23:02:11.694522 9335 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:11.695515 9335 heartbeater.cc:511] Master 127.24.31.254:39623 requested a full tablet report, sending...
I20260809 23:02:11.696043 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.680449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54568 (local address 127.24.31.254:39623)
0809 23:02:11.681548 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.681552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.681566 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:02:11.681605 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.681608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.681669 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.681795 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:11.683365 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.684181 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.685693 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.686579 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.686699 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.690004 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:11.690011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:11.690018 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:11.690051 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:11.692905 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:11.693835 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:11.693838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:11.693840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:11.693979 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:11.694452 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:11.694458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:11.694464 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:11.694827 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:11.695084 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.695400 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.695624 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":892,"threads_started":1}
I20260809 23:02:11.697402 8805 ts_manager.cc:194] Registered new tserver with Master: 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789)
I20260809 23:02:11.698287 8805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:48249
I20260809 23:02:11.699787 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:02:11.706202 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.700802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54570 (local address 127.24.31.254:39623)
0809 23:02:11.700962 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.700966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.701413 (+ 447us) server_negotiation.cc:407] Connection header received
0809 23:02:11.701569 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.701572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.701696 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.701808 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:02:11.702163 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.702807 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.704208 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.704616 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.704703 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.705265 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:02:11.705718 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:02:11.705823 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.705955 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.706016 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 23:02:11.707257 8803 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:02:11.714788 8803 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:02:11.754578 9348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.726209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:41409 (local address 127.0.0.1:50620)
0809 23:02:11.737892 (+ 11683us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:11.738786 (+ 894us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:11.739017 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:11.741271 (+ 2254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:11.741282 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:11.741326 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:11.741926 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.741941 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.745876 (+ 3935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.745884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.748892 (+ 3008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.748912 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.753499 (+ 4587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.753508 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.753730 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.753786 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 23:02:11.753914 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 23:02:11.754127 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11362,"thread_start_us":27794,"threads_started":1}
I20260809 23:02:11.754936 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.726493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.24.31.194:41409)
0809 23:02:11.740505 (+ 14012us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.740517 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.740560 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:02:11.740674 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.740685 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.740971 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.742380 (+ 1409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:11.742626 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.745022 (+ 2396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.749853 (+ 4831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.751701 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.751862 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.752103 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.754390 (+ 2287us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.754588 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13783,"thread_start_us":15593,"threads_started":1}
I20260809 23:02:11.759238 9105 tablet_service.cc:1467] Processing CreateTablet for tablet 871a6d2a0b0a4857aa396b28a362eb54 (DEFAULT_TABLE table=test-table [id=7a5d5504ddd441619acaec0b4f381ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:11.760506 9105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 871a6d2a0b0a4857aa396b28a362eb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:11.762750 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.727319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.24.31.193:38061)
0809 23:02:11.753865 (+ 26546us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.753875 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.753905 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:02:11.754011 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.754019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.754253 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.754775 (+ 522us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:11.755557 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.756830 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.759323 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.760899 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.761078 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.761356 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.762172 (+ 816us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.762376 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26383,"thread_start_us":26213,"threads_started":1}
I20260809 23:02:11.762938 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.726486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60208)
0809 23:02:11.752983 (+ 26497us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:11.753004 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:11.753127 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:11.754475 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:11.754480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:11.754507 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:11.755368 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.755379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.757001 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.757006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.759128 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.759147 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.761141 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.761171 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.761400 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.761438 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:02:11.761580 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 23:02:11.761761 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26358,"spinlock_wait_cycles":930688,"thread_start_us":14687,"threads_started":1}
I20260809 23:02:11.763685 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.726816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:38789 (local address 127.0.0.1:38960)
0809 23:02:11.754342 (+ 27526us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:11.754362 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:11.755010 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:11.756154 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:11.756161 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:11.756189 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:11.756559 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.756573 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.758497 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.758505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.760565 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.760582 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.762739 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.762746 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.762920 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.762955 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 23:02:11.763051 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 23:02:11.763191 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27365,"thread_start_us":26412,"threads_started":1}
I20260809 23:02:11.764213 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.727562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.24.31.195:38789)
0809 23:02:11.752348 (+ 24786us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.752356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.754467 (+ 2111us) server_negotiation.cc:407] Connection header received
0809 23:02:11.755259 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.755268 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.755522 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.755693 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:11.756837 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.758354 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.760752 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.762186 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.762326 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.762575 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.763391 (+ 816us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.763824 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24601,"thread_start_us":24781,"threads_started":1}
I20260809 23:02:11.766426 8950 tablet_service.cc:1467] Processing CreateTablet for tablet 871a6d2a0b0a4857aa396b28a362eb54 (DEFAULT_TABLE table=test-table [id=7a5d5504ddd441619acaec0b4f381ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:11.767791 8950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 871a6d2a0b0a4857aa396b28a362eb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:11.767568 9260 tablet_service.cc:1467] Processing CreateTablet for tablet 871a6d2a0b0a4857aa396b28a362eb54 (DEFAULT_TABLE table=test-table [id=7a5d5504ddd441619acaec0b4f381ebc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:11.768875 9260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 871a6d2a0b0a4857aa396b28a362eb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:11.775985 9030 heartbeater.cc:503] Master 127.24.31.254:39623 was elected leader, sending a full tablet report...
I20260809 23:02:11.778472 9355 tablet_bootstrap.cc:492] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: Bootstrap starting.
I20260809 23:02:11.782481 9355 tablet_bootstrap.cc:654] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:11.782896 9356 tablet_bootstrap.cc:492] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: Bootstrap starting.
I20260809 23:02:11.784615 9355 log.cc:824] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:11.786456 9356 tablet_bootstrap.cc:654] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:11.787223 9357 tablet_bootstrap.cc:492] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Bootstrap starting.
I20260809 23:02:11.787787 9356 log.cc:824] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:11.788435 9355 tablet_bootstrap.cc:492] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: No bootstrap required, opened a new log
I20260809 23:02:11.788820 9355 ts_tablet_manager.cc:1397] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 23:02:11.790380 9356 tablet_bootstrap.cc:492] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: No bootstrap required, opened a new log
I20260809 23:02:11.790655 9356 ts_tablet_manager.cc:1397] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 23:02:11.791020 9357 tablet_bootstrap.cc:654] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:11.792320 9357 log.cc:824] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:11.795570 9357 tablet_bootstrap.cc:492] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: No bootstrap required, opened a new log
I20260809 23:02:11.795853 9357 ts_tablet_manager.cc:1397] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 23:02:11.798662 9356 raft_consensus.cc:348] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:11.799016 9356 raft_consensus.cc:374] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:11.799141 9356 raft_consensus.cc:729] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c56cc956994e43b7b56f91f659c46f, State: Initialized, Role: FOLLOWER
I20260809 23:02:11.799753 9356 consensus_queue.cc:260] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:11.800895 9335 heartbeater.cc:503] Master 127.24.31.254:39623 was elected leader, sending a full tablet report...
I20260809 23:02:11.800796 9355 raft_consensus.cc:348] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:11.801262 9355 raft_consensus.cc:374] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:11.801362 9355 raft_consensus.cc:729] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3b583f140284b668f7d2faab8b1c954, State: Initialized, Role: FOLLOWER
I20260809 23:02:11.801553 9356 ts_tablet_manager.cc:1428] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 23:02:11.801975 9355 consensus_queue.cc:260] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:11.803155 9187 heartbeater.cc:503] Master 127.24.31.254:39623 was elected leader, sending a full tablet report...
I20260809 23:02:11.803808 9355 ts_tablet_manager.cc:1428] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260809 23:02:11.807065 9357 raft_consensus.cc:348] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:11.807590 9357 raft_consensus.cc:374] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:11.807737 9357 raft_consensus.cc:729] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9fdbedf577c4f57acd051353e3383c5, State: Initialized, Role: FOLLOWER
I20260809 23:02:11.808416 9357 consensus_queue.cc:260] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:11.810139 9357 ts_tablet_manager.cc:1428] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260809 23:02:11.873103 9361 raft_consensus.cc:482] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:11.873463 9361 raft_consensus.cc:504] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:11.875177 9361 leader_election.cc:288] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409), d9fdbedf577c4f57acd051353e3383c5 (127.24.31.193:38061)
I20260809 23:02:11.881170 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.875749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:55333 (local address 127.24.31.193:38061)
0809 23:02:11.875980 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.875987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.876147 (+ 160us) server_negotiation.cc:407] Connection header received
0809 23:02:11.876401 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.876406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.876474 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.876582 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:11.876988 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.877846 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.879525 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.880260 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.880451 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.880533 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.880782 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.880905 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 23:02:11.881202 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.875676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.24.31.195:55333)
0809 23:02:11.876054 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:11.876071 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:11.876221 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:11.876608 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:11.876613 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:11.876641 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:11.876819 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.876844 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.878013 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.878018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.879360 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.879372 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.880418 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.880425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.880617 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.880641 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 23:02:11.880744 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 23:02:11.880871 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":37}
I20260809 23:02:11.881801 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.875682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:48725 (local address 127.24.31.194:41409)
0809 23:02:11.875987 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.875995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.876667 (+ 672us) server_negotiation.cc:407] Connection header received
0809 23:02:11.876944 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.876948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.877004 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.877102 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:11.877685 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.878375 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.880025 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.880787 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.880970 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.881043 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.881443 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.881565 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260809 23:02:11.882215 8974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "871a6d2a0b0a4857aa396b28a362eb54" candidate_uuid: "81c56cc956994e43b7b56f91f659c46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9fdbedf577c4f57acd051353e3383c5" is_pre_election: true
I20260809 23:02:11.882382 9364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.875713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:41409 (local address 127.24.31.195:48725)
0809 23:02:11.876540 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:11.876554 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:11.876759 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:11.877121 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:11.877125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:11.877174 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:11.877509 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.877517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.878528 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.878533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.879870 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:11.879881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.880988 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.880994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:11.881226 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.881278 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 23:02:11.881986 (+ 708us) client_negotiation.cc:240] Negotiation successful
0809 23:02:11.882109 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":1,"spinlock_wait_cycles":24320,"thread_start_us":456,"threads_started":1}
I20260809 23:02:11.882726 8974 raft_consensus.cc:2454] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81c56cc956994e43b7b56f91f659c46f in term 0.
I20260809 23:02:11.883404 9129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "871a6d2a0b0a4857aa396b28a362eb54" candidate_uuid: "81c56cc956994e43b7b56f91f659c46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3b583f140284b668f7d2faab8b1c954" is_pre_election: true
I20260809 23:02:11.883494 9208 leader_election.cc:302] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81c56cc956994e43b7b56f91f659c46f, d9fdbedf577c4f57acd051353e3383c5; no voters:
I20260809 23:02:11.883836 9129 raft_consensus.cc:2454] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81c56cc956994e43b7b56f91f659c46f in term 0.
I20260809 23:02:11.883852 9361 raft_consensus.cc:2781] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:02:11.884011 9361 raft_consensus.cc:482] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:02:11.884123 9361 raft_consensus.cc:3037] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:11.886715 9361 raft_consensus.cc:504] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:11.887293 9361 leader_election.cc:288] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [CANDIDATE]: Term 1 election: Requested vote from peers a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409), d9fdbedf577c4f57acd051353e3383c5 (127.24.31.193:38061)
I20260809 23:02:11.887786 8974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "871a6d2a0b0a4857aa396b28a362eb54" candidate_uuid: "81c56cc956994e43b7b56f91f659c46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9fdbedf577c4f57acd051353e3383c5"
I20260809 23:02:11.887775 9129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "871a6d2a0b0a4857aa396b28a362eb54" candidate_uuid: "81c56cc956994e43b7b56f91f659c46f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3b583f140284b668f7d2faab8b1c954"
I20260809 23:02:11.887985 9129 raft_consensus.cc:3037] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:11.887995 8974 raft_consensus.cc:3037] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:11.891286 8974 raft_consensus.cc:2454] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81c56cc956994e43b7b56f91f659c46f in term 1.
I20260809 23:02:11.891286 9129 raft_consensus.cc:2454] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81c56cc956994e43b7b56f91f659c46f in term 1.
I20260809 23:02:11.891691 9208 leader_election.cc:302] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81c56cc956994e43b7b56f91f659c46f, d9fdbedf577c4f57acd051353e3383c5; no voters:
I20260809 23:02:11.891989 9361 raft_consensus.cc:2781] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:02:11.892431 9361 raft_consensus.cc:686] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 1 LEADER]: Becoming Leader. State: Replica: 81c56cc956994e43b7b56f91f659c46f, State: Running, Role: LEADER
I20260809 23:02:11.893078 9361 consensus_queue.cc:237] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:11.898240 8805 catalog_manager.cc:5495] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f reported cstate change: term changed from 0 to 1, leader changed from <none> to 81c56cc956994e43b7b56f91f659c46f (127.24.31.195). New cstate: current_term: 1 leader_uuid: "81c56cc956994e43b7b56f91f659c46f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } health_report { overall_health: HEALTHY } } }
W20260809 23:02:11.921307 9336 tablet.cc:2369] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:02:11.948279 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.942840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38972 (local address 127.24.31.195:38789)
0809 23:02:11.943056 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.943062 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.943082 (+ 20us) server_negotiation.cc:407] Connection header received
0809 23:02:11.943332 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.943337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.943412 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.943570 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:02:11.943961 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.944667 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.946149 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.946583 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:11.946723 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:11.947200 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:02:11.947681 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:02:11.947841 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:11.947900 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 23:02:11.948006 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 23:02:11.956323 9129 raft_consensus.cc:1261] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [term 1 FOLLOWER]: Refusing update from remote peer 81c56cc956994e43b7b56f91f659c46f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:02:11.956372 8974 raft_consensus.cc:1261] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Refusing update from remote peer 81c56cc956994e43b7b56f91f659c46f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 23:02:11.957432 9189 tablet.cc:2369] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:02:11.957422 9365 consensus_queue.cc:1035] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:02:11.957825 9361 consensus_queue.cc:1035] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:02:11.971406 9370 mvcc.cc:205] Tried to move back new op lower bound from 7316752514883239936 to 7316752514638540800. Current Snapshot: MvccSnapshot[applied={T|T < 7316752514883239936}]
I20260809 23:02:11.973529 9369 mvcc.cc:205] Tried to move back new op lower bound from 7316752514883239936 to 7316752514638540800. Current Snapshot: MvccSnapshot[applied={T|T < 7316752514883239936}]
I20260809 23:02:11.986052 8805 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:02:11.986378 8805 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:02:11.990233 8805 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Sending DeleteTablet for 3 replicas of tablet 871a6d2a0b0a4857aa396b28a362eb54
I20260809 23:02:11.991428 8950 tablet_service.cc:1514] Processing DeleteTablet for tablet 871a6d2a0b0a4857aa396b28a362eb54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:11 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:60208
I20260809 23:02:11.991439 9260 tablet_service.cc:1514] Processing DeleteTablet for tablet 871a6d2a0b0a4857aa396b28a362eb54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:11 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:38960
I20260809 23:02:11.991434 9105 tablet_service.cc:1514] Processing DeleteTablet for tablet 871a6d2a0b0a4857aa396b28a362eb54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:11 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:50620
I20260809 23:02:11.994037 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 8855
I20260809 23:02:12.017079 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:11.995004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.24.31.193:38061)
0809 23:02:11.995180 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:11.995187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:11.995289 (+ 102us) server_negotiation.cc:407] Connection header received
0809 23:02:11.995529 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:11.995538 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:11.995645 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:11.995756 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:11.998075 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:11.999654 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:12.002294 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:12.003260 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:12.003393 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:12.008013 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:12.008038 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:12.008065 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:12.008127 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:12.013312 (+ 5185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:12.014397 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:12.014413 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:12.014419 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:12.014635 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:12.015106 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:12.015114 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:12.015120 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:12.015745 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:12.016084 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:12.016410 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 23:02:12.016659 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:02:12.017596 9378 tablet_replica.cc:330] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: stopping tablet replica
I20260809 23:02:12.018239 9025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:12.018930 9378 raft_consensus.cc:2229] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:02:12.019711 9378 raft_consensus.cc:2258] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:12.021636 9381 tablet_replica.cc:330] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: stopping tablet replica
I20260809 23:02:12.022243 9381 raft_consensus.cc:2229] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:02:12.023049 9381 raft_consensus.cc:2258] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:12.025419 9378 ts_tablet_manager.cc:1905] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:12.025456 9381 ts_tablet_manager.cc:1905] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:12.032593 9378 ts_tablet_manager.cc:1918] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:12.033535 9378 log.cc:1196] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/871a6d2a0b0a4857aa396b28a362eb54
I20260809 23:02:12.035074 9381 ts_tablet_manager.cc:1918] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:12.035162 9378 ts_tablet_manager.cc:1939] T 871a6d2a0b0a4857aa396b28a362eb54 P a3b583f140284b668f7d2faab8b1c954: Deleting consensus metadata
I20260809 23:02:12.035279 9381 log.cc:1196] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/871a6d2a0b0a4857aa396b28a362eb54
I20260809 23:02:12.035864 9381 ts_tablet_manager.cc:1939] T 871a6d2a0b0a4857aa396b28a362eb54 P 81c56cc956994e43b7b56f91f659c46f: Deleting consensus metadata
I20260809 23:02:12.037660 8790 catalog_manager.cc:4841] TS 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789): tablet 871a6d2a0b0a4857aa396b28a362eb54 (table test-table [id=7a5d5504ddd441619acaec0b4f381ebc]) successfully deleted
I20260809 23:02:12.038447 8788 catalog_manager.cc:4841] TS a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409): tablet 871a6d2a0b0a4857aa396b28a362eb54 (table test-table [id=7a5d5504ddd441619acaec0b4f381ebc]) successfully deleted
I20260809 23:02:12.125681 9379 tablet_replica.cc:330] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: stopping tablet replica
I20260809 23:02:12.126231 9379 raft_consensus.cc:2229] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:02:12.126742 9379 raft_consensus.cc:2258] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:12.126971 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 8855
W20260809 23:02:12.147056 8787 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.31.193:38061 (error 108)
I20260809 23:02:12.147833 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 9034
I20260809 23:02:12.149391 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.148816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60230)
0809 23:02:12.149083 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:12.149228 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
W20260809 23:02:12.149775 8787 catalog_manager.cc:4584] TS d9fdbedf577c4f57acd051353e3383c5 (127.24.31.193:38061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 871a6d2a0b0a4857aa396b28a362eb54: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
I20260809 23:02:12.171141 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.148752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50632 (local address 127.24.31.194:41409)
0809 23:02:12.148959 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:12.148967 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:12.149050 (+ 83us) server_negotiation.cc:407] Connection header received
0809 23:02:12.149296 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:12.149301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:12.149378 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:12.149494 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:12.151024 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:12.151773 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:12.153216 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:12.153639 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:12.153784 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:12.165041 (+ 11257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:12.165061 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:12.165086 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:12.165127 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:12.168086 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:12.169009 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:12.169016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:12.169018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:12.169201 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:12.169618 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:12.169621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:12.169623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:12.170012 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:12.170234 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:12.170659 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 23:02:12.170824 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 23:02:12.171881 9182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:12.203487 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.202992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60238)
0809 23:02:12.203202 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:12.203322 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 23:02:12.204573 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.204212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60246)
0809 23:02:12.204361 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:12.204459 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 23:02:12.265297 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.264707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60252)
0809 23:02:12.264960 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:12.265090 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 23:02:12.266502 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.266083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60254)
0809 23:02:12.266256 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:12.266348 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 23:02:12.302728 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 9034
I20260809 23:02:12.323201 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 9191
I20260809 23:02:12.340169 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.324014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38980 (local address 127.24.31.195:38789)
0809 23:02:12.324207 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:12.324213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:12.324274 (+ 61us) server_negotiation.cc:407] Connection header received
0809 23:02:12.324436 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:12.324442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:12.324527 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:12.324658 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:12.326130 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:12.327476 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:12.328778 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:12.329485 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:12.329622 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:12.332688 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:12.332706 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:12.332727 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:12.332779 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:12.336833 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:12.337723 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:12.337739 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:12.337754 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:12.337952 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:12.338479 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:12.338484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:12.338488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:12.338970 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:12.339301 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:12.339523 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 23:02:12.339749 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 23:02:12.341048 9330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:12.347236 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.346816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60256)
0809 23:02:12.347006 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:12.347088 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 23:02:12.348266 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.347892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60266)
0809 23:02:12.348087 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:12.348162 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 23:02:12.466512 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 9191
I20260809 23:02:12.488483 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 8767
I20260809 23:02:12.489832 8847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:12.605213 9346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:12.604500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:38061 (local address 127.0.0.1:60282)
0809 23:02:12.604743 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:12.604886 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:38061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 23:02:12.605494 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 8767
I20260809 23:02:12.630990 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39623
--webserver_interface=127.24.31.254
--webserver_port=45991
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:12.967039 9388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:12.967319 9388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:12.967500 9388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:12.974753 9388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:02:12.974905 9388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:12.974996 9388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:12.975098 9388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:02:12.975183 9388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:02:12.985116 9388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=45991
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:12.986568 9388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:12.987941 9388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:13.000051 9394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:13.000022 9395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:13.000344 9397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:13.000974 9388 server_base.cc:1034] running on GCE node
I20260809 23:02:13.001840 9388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:13.003216 9388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:13.004452 9388 hybrid_clock.cc:648] HybridClock initialized: now 1786316533004400 us; error 61 us; skew 500 ppm
I20260809 23:02:13.004808 9388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:13 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316533, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:13.007893 9388 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:02:13.009433 9388 webserver.cc:460] Webserver started at http://127.24.31.254:45991/ using document root <none> and password file <none>
I20260809 23:02:13.010183 9388 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:13.010350 9388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:13.016023 9388 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 23:02:13.018965 9404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:13.020401 9388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:02:13.020612 9388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "8dee14416d0c45a48be82c9cc5bbea80"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "d0c93994f80da38ad3f8765e6a456c63"
server_key_iv: "597023065a5a75cd211fa03c09b5f60a"
server_key_version: "encryptionkey@0"
I20260809 23:02:13.021723 9388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:13.055176 9388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:13.059559 9388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:13.059871 9388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:13.079187 9388 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:39623
I20260809 23:02:13.079195 9463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:39623 every 8 connection(s)
I20260809 23:02:13.080802 9388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:02:13.086803 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 9388
I20260809 23:02:13.087672 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:38061
--local_ip_for_outbound_sockets=127.24.31.193
--tserver_master_addrs=127.24.31.254:39623
--webserver_port=38925
--webserver_interface=127.24.31.193
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:02:13.087724 9464 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:02:13.091053 9464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Bootstrap starting.
I20260809 23:02:13.115860 9464 log.cc:824] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:13.140012 9464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:02:13.140839 9464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80: Bootstrap complete.
I20260809 23:02:13.155148 9464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:13.156020 9464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dee14416d0c45a48be82c9cc5bbea80, State: Initialized, Role: FOLLOWER
I20260809 23:02:13.156728 9464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:13.156975 9464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:02:13.157092 9464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:02:13.157321 9464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 2 FOLLOWER]: Advancing to term 3
I20260809 23:02:13.161087 9464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:13.161640 9464 leader_election.cc:302] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dee14416d0c45a48be82c9cc5bbea80; no voters:
I20260809 23:02:13.162387 9464 leader_election.cc:288] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 23:02:13.162523 9469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 23:02:13.165033 9464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:02:13.169502 9469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [term 3 LEADER]: Becoming Leader. State: Replica: 8dee14416d0c45a48be82c9cc5bbea80, State: Running, Role: LEADER
I20260809 23:02:13.170159 9469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } }
I20260809 23:02:13.179270 9471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dee14416d0c45a48be82c9cc5bbea80. Latest consensus state: current_term: 3 leader_uuid: "8dee14416d0c45a48be82c9cc5bbea80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } } }
I20260809 23:02:13.179636 9471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:13.180208 9470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8dee14416d0c45a48be82c9cc5bbea80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dee14416d0c45a48be82c9cc5bbea80" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39623 } } }
I20260809 23:02:13.180539 9470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dee14416d0c45a48be82c9cc5bbea80 [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:13.180621 9486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:02:13.199863 9486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:02:13.201637 9486 catalog_manager.cc:1259] Loaded cluster ID: 0e4ab2ce44054bb5af41de6cc5207ebf
I20260809 23:02:13.201864 9486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 23:02:13.204015 9486 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4430a4b9ad google::LogMessage::Fail() at ??:0
I20260809 23:02:13.207540 9490 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f4430a4e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f4430a4b4de google::LogMessage::Flush() at ??:0
@ 0x7f4430a4ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f444647e512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f444647b976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4446503610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f4446503439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4433697345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f443369a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f443369a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f4433675eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f443694f6db start_thread at ??:0
@ 0x7f442f8e561f clone at ??:0
W20260809 23:02:13.476307 9467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:13.476663 9467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:13.476824 9467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:13.484340 9467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:13.484508 9467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:13.484683 9467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:02:13.494801 9467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:38061
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=38925
--tserver_master_addrs=127.24.31.254:39623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:13.496196 9467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:13.497637 9467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:13.511437 9515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:13.511405 9518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:13.511893 9516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:13.512678 9467 server_base.cc:1034] running on GCE node
I20260809 23:02:13.513378 9467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:13.514333 9467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:13.515616 9467 hybrid_clock.cc:648] HybridClock initialized: now 1786316533515554 us; error 51 us; skew 500 ppm
I20260809 23:02:13.515947 9467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:13 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316533, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:13.519840 9467 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:02:13.521359 9467 webserver.cc:460] Webserver started at http://127.24.31.193:38925/ using document root <none> and password file <none>
I20260809 23:02:13.522023 9467 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:13.522130 9467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:13.527545 9467 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 23:02:13.530570 9526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:13.531903 9467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:13.532096 9467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "d9fdbedf577c4f57acd051353e3383c5"
format_stamp: "Formatted at 2026-08-09 23:02:07 on dist-test-slave-vxj2"
server_key: "3b5cea9003e660ab3466ebb79c9972fd"
server_key_iv: "904a588d54dc1d53ce824ef45ee35afd"
server_key_version: "encryptionkey@0"
I20260809 23:02:13.533056 9467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:13.562950 9467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:13.567112 9467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:13.567411 9467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:13.568524 9467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:13.570513 9535 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 23:02:13.576069 9467 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 23:02:13.576221 9467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 23:02:13.576376 9467 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 23:02:13.579560 9467 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 23:02:13.579672 9467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260809 23:02:13.580101 9535 tablet_bootstrap.cc:492] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Bootstrap starting.
I20260809 23:02:13.609200 9535 log.cc:824] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:13.622556 9535 tablet_bootstrap.cc:492] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:02:13.623340 9535 tablet_bootstrap.cc:492] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Bootstrap complete.
I20260809 23:02:13.624079 9467 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:38061
I20260809 23:02:13.624099 9655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:38061 every 8 connection(s)
I20260809 23:02:13.624850 9535 ts_tablet_manager.cc:1397] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Time spent bootstrapping tablet: real 0.045s user 0.020s sys 0.020s
I20260809 23:02:13.625463 9467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:02:13.629959 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 9467
I20260809 23:02:13.630753 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:41409
--local_ip_for_outbound_sockets=127.24.31.194
--tserver_master_addrs=127.24.31.254:39623
--webserver_port=36411
--webserver_interface=127.24.31.194
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:02:13.630829 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:13.628521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:37559)
0809 23:02:13.629522 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:13.629888 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":434,"threads_started":1}
W20260809 23:02:13.631580 9656 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:39623 (0 consecutive failures): Network error: Failed to ping master at 127.24.31.254:39623: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
I20260809 23:02:13.632441 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:13.632114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:56143)
0809 23:02:13.632262 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:13.632338 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 23:02:13.633447 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:13.633023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:53737)
0809 23:02:13.633230 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:13.633305 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 23:02:13.633754 9656 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:02:13.634946 9535 raft_consensus.cc:348] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:13.636319 9535 raft_consensus.cc:729] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9fdbedf577c4f57acd051353e3383c5, State: Initialized, Role: FOLLOWER
I20260809 23:02:13.636968 9535 consensus_queue.cc:260] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:13.638579 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:13.638257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:52217)
0809 23:02:13.638395 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:13.638479 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 23:02:13.638705 9535 ts_tablet_manager.cc:1428] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5: Time spent starting tablet: real 0.014s user 0.013s sys 0.003s
I20260809 23:02:13.639531 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:13.639229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:48995)
0809 23:02:13.639359 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:13.639427 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 23:02:13.964162 9660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:13.964432 9660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:13.964629 9660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:13.971966 9660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:13.972096 9660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:13.972296 9660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:02:13.981876 9660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:41409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=36411
--tserver_master_addrs=127.24.31.254:39623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:13.983208 9660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:13.984572 9660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:13.997812 9665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:13.997758 9666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:13.997840 9669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:13.998594 9660 server_base.cc:1034] running on GCE node
I20260809 23:02:13.999198 9660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:14.000149 9660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:14.001335 9660 hybrid_clock.cc:648] HybridClock initialized: now 1786316534001294 us; error 33 us; skew 500 ppm
I20260809 23:02:14.001662 9660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:14 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316534, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:14.004817 9660 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:02:14.006419 9660 webserver.cc:460] Webserver started at http://127.24.31.194:36411/ using document root <none> and password file <none>
I20260809 23:02:14.007136 9660 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:14.007287 9660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:14.012574 9660 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 23:02:14.015254 9680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:14.016530 9660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:02:14.016713 9660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "a3b583f140284b668f7d2faab8b1c954"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "c4b51bf421affe0655d76fe6c6347e37"
server_key_iv: "69a170623aa1eef295eb0300daee95fd"
server_key_version: "encryptionkey@0"
I20260809 23:02:14.017763 9660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:14.038589 9660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:14.042817 9660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:14.043125 9660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:14.044144 9660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:14.045629 9660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:14.045766 9660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:14.045902 9660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:14.046013 9660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:14.080852 9660 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:41409
I20260809 23:02:14.080945 9810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:41409 every 8 connection(s)
I20260809 23:02:14.082376 9660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:02:14.087102 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:14.085347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:38757)
0809 23:02:14.086058 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:14.086269 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":266,"threads_started":1}
W20260809 23:02:14.087736 9811 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:39623 (0 consecutive failures): Network error: Failed to ping master at 127.24.31.254:39623: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
I20260809 23:02:14.088536 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 9660
I20260809 23:02:14.088591 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:14.088193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:45347)
0809 23:02:14.088368 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:14.088453 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 23:02:14.089227 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:38789
--local_ip_for_outbound_sockets=127.24.31.195
--tserver_master_addrs=127.24.31.254:39623
--webserver_port=38607
--webserver_interface=127.24.31.195
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:46571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:02:14.089551 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:14.089105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:45575)
0809 23:02:14.089296 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:14.089398 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260809 23:02:14.089849 9811 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:02:14.424939 9814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:14.425336 9814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:14.425529 9814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:14.432909 9814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:14.433084 9814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:14.433286 9814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:02:14.443825 9814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:46571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:38789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=38607
--tserver_master_addrs=127.24.31.254:39623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:14.445236 9814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:14.446641 9814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:14.460127 9822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:14.460155 9819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:14.460212 9820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:14.460981 9814 server_base.cc:1034] running on GCE node
I20260809 23:02:14.461659 9814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:14.462641 9814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:14.463872 9814 hybrid_clock.cc:648] HybridClock initialized: now 1786316534463814 us; error 68 us; skew 500 ppm
I20260809 23:02:14.464190 9814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:14 dist-test-slave-vxj2 krb5kdc[8101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316534, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:14.467440 9814 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:02:14.468878 9814 webserver.cc:460] Webserver started at http://127.24.31.195:38607/ using document root <none> and password file <none>
I20260809 23:02:14.469774 9814 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:14.469930 9814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:14.475499 9814 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 23:02:14.478595 9831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:14.479982 9814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:14.480182 9814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "81c56cc956994e43b7b56f91f659c46f"
format_stamp: "Formatted at 2026-08-09 23:02:08 on dist-test-slave-vxj2"
server_key: "83635f83dd31552fcdeae5f9e79ce34f"
server_key_iv: "06a0592e4b3b526cf9d4825b2a36e946"
server_key_version: "encryptionkey@0"
I20260809 23:02:14.481240 9814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:14.504860 9814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:14.509296 9814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:14.509606 9814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:14.510676 9814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:14.512132 9814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:14.512271 9814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:14.512429 9814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:14.512530 9814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:14.547477 9814 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:38789
I20260809 23:02:14.547569 9961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:38789 every 8 connection(s)
I20260809 23:02:14.549002 9814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:02:14.553905 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:14.551993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:33763)
0809 23:02:14.552714 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:14.552968 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":280,"threads_started":1}
W20260809 23:02:14.554762 9962 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:39623 (0 consecutive failures): Network error: Failed to ping master at 127.24.31.254:39623: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
I20260809 23:02:14.555287 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 9814
I20260809 23:02:14.555913 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:14.555539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:34897)
0809 23:02:14.555707 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:14.555783 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 23:02:14.556828 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:14.556481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:38693)
0809 23:02:14.556626 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:14.556715 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 23:02:14.557200 9962 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:02:14.641089 9966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:14.640385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:35257)
0809 23:02:14.640825 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:14.640902 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":215,"threads_started":1}
I20260809 23:02:15.091152 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:15.090379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:60257)
0809 23:02:15.090807 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:15.090917 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":210,"threads_started":1}
I20260809 23:02:15.196844 9968 raft_consensus.cc:482] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:15.197118 9968 raft_consensus.cc:504] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:15.199124 9968 leader_election.cc:288] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409), 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789)
Aug 09 23:02:15 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316533, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
Aug 09 23:02:15 dist-test-slave-vxj2 krb5kdc[8101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316533, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 23:02:15.227540 9971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:15.199684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:59451 (local address 127.24.31.194:41409)
0809 23:02:15.202168 (+ 2484us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:15.202184 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:15.202229 (+ 45us) server_negotiation.cc:407] Connection header received
0809 23:02:15.204575 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:15.204585 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:15.205029 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:15.205421 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:15.208430 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:15.210365 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:15.213210 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:15.214278 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:15.214476 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:15.219422 (+ 4946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:15.219435 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:15.219457 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:15.219517 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:15.223301 (+ 3784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:15.224076 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:15.224085 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:15.224092 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:15.224293 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:15.224848 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:15.224852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:15.224855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:15.225618 (+ 763us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:15.225806 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:15.226680 (+ 874us) server_negotiation.cc:299] Negotiation successful
0809 23:02:15.227136 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2276,"thread_start_us":2102,"threads_started":1}
I20260809 23:02:15.227624 9973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:15.199823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:55427 (local address 127.24.31.195:38789)
0809 23:02:15.202278 (+ 2455us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:15.202290 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:15.202318 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:02:15.204514 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:15.204528 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:15.205029 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:15.205421 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:15.208430 (+ 3009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:15.210365 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:15.213210 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:15.214277 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:15.214478 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:15.219703 (+ 5225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:15.219713 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:15.219728 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:15.219774 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:15.224064 (+ 4290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:15.224834 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:15.224842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:15.224849 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:15.225045 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:15.225492 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:15.225497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:15.225500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:15.226068 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:15.226198 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:15.226847 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 23:02:15.227210 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2303,"thread_start_us":2198,"threads_started":1}
I20260809 23:02:15.227602 9969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:15.199447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:38789 (local address 127.24.31.193:55427)
0809 23:02:15.201694 (+ 2247us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:15.201716 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:15.201829 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:15.205617 (+ 3788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:15.205630 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:15.206919 (+ 1289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:15.208206 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:15.208234 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:15.210590 (+ 2356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:15.210596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:15.213017 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:15.213033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:15.214547 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:15.214554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:15.214896 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:15.215919 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:15.215951 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:15.219380 (+ 3429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:15.224199 (+ 4819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:15.224202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:15.224205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:15.224697 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:15.225233 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:15.225236 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:15.225239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:15.225378 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:15.226211 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:15.226216 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:15.226390 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 23:02:15.226586 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 23:02:15.227058 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2123,"mutex_wait_us":437,"spinlock_wait_cycles":57856,"thread_start_us":604,"threads_started":1}
I20260809 23:02:15.227627 9970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:15.199393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:41409 (local address 127.24.31.193:59451)
0809 23:02:15.200110 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:15.200156 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:15.200879 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:15.205617 (+ 4738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:15.205630 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:15.206919 (+ 1289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:15.208206 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:15.208234 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:15.210594 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:15.210598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:15.213017 (+ 2419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:15.213033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:15.214561 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:15.214567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:15.214891 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:15.215938 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:15.215964 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:15.219094 (+ 3130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:15.223461 (+ 4367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:15.223468 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:15.223480 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:15.223930 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:15.224415 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:15.224420 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:15.224423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:15.224686 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:15.225788 (+ 1102us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:15.225795 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:15.226074 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 23:02:15.226415 (+ 341us) client_negotiation.cc:240] Negotiation successful
0809 23:02:15.227176 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":597,"spinlock_wait_cycles":20224,"thread_start_us":2019,"threads_started":1}
W20260809 23:02:15.231717 9532 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
W20260809 23:02:15.232018 9530 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
I20260809 23:02:15.232180 9530 leader_election.cc:302] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9fdbedf577c4f57acd051353e3383c5; no voters: 81c56cc956994e43b7b56f91f659c46f, a3b583f140284b668f7d2faab8b1c954
I20260809 23:02:15.232604 9968 raft_consensus.cc:2726] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:02:15.558580 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:15.557731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:48495)
0809 23:02:15.558200 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:15.558349 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":241,"threads_started":1}
I20260809 23:02:15.642686 9970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:15.642087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:49759)
0809 23:02:15.642360 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:15.642500 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 23:02:16.092635 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:16.091990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:46369)
0809 23:02:16.092385 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:16.092485 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":217,"threads_started":1}
I20260809 23:02:16.560468 9976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:16.559691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:50925)
0809 23:02:16.560231 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:16.560311 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":267,"threads_started":1}
I20260809 23:02:16.644500 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:16.643682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:34887)
0809 23:02:16.644204 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:16.644315 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":242,"threads_started":1}
I20260809 23:02:17.094345 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:17.093600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:60767)
0809 23:02:17.094069 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:17.094146 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":256,"threads_started":1}
I20260809 23:02:17.323947 9979 raft_consensus.cc:482] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:17.324254 9979 raft_consensus.cc:504] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:17.325022 9979 leader_election.cc:288] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409), 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789)
W20260809 23:02:17.325999 9530 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
W20260809 23:02:17.326231 9532 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
I20260809 23:02:17.326391 9532 leader_election.cc:302] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9fdbedf577c4f57acd051353e3383c5; no voters: 81c56cc956994e43b7b56f91f659c46f, a3b583f140284b668f7d2faab8b1c954
I20260809 23:02:17.326707 9979 raft_consensus.cc:2726] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:02:17.562222 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:17.561458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:55243)
0809 23:02:17.561958 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:17.562042 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":280,"threads_started":1}
I20260809 23:02:17.646283 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:17.645407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:36409)
0809 23:02:17.645917 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:17.646058 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":243,"threads_started":1}
I20260809 23:02:18.095877 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:18.095152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:37505)
0809 23:02:18.095578 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:18.095657 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":212,"threads_started":1}
I20260809 23:02:18.563941 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:18.563184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:33281)
0809 23:02:18.563674 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:18.563756 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":266,"threads_started":1}
I20260809 23:02:18.648094 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:18.647390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:59047)
0809 23:02:18.647833 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:18.647916 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":228,"threads_started":1}
I20260809 23:02:19.097513 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:19.096763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:37649)
0809 23:02:19.097256 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:19.097342 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":245,"threads_started":1}
I20260809 23:02:19.565779 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:19.565001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:55315)
0809 23:02:19.565538 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:19.565619 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":350,"threads_started":1}
I20260809 23:02:19.649704 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:19.648954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:57987)
0809 23:02:19.649464 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:19.649543 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":326,"threads_started":1}
I20260809 23:02:20.066439 9990 raft_consensus.cc:482] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:20.066681 9990 raft_consensus.cc:504] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:20.067361 9990 leader_election.cc:288] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409), 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789)
W20260809 23:02:20.068317 9532 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
W20260809 23:02:20.068537 9530 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
I20260809 23:02:20.068698 9530 leader_election.cc:302] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9fdbedf577c4f57acd051353e3383c5; no voters: 81c56cc956994e43b7b56f91f659c46f, a3b583f140284b668f7d2faab8b1c954
I20260809 23:02:20.069000 9990 raft_consensus.cc:2726] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:02:20.099143 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:20.098342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:33089)
0809 23:02:20.098850 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:20.098937 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":277,"threads_started":1}
I20260809 23:02:20.567399 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:20.566669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:58201)
0809 23:02:20.567148 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:20.567230 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":268,"threads_started":1}
I20260809 23:02:20.651358 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:20.650581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:52201)
0809 23:02:20.651041 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:20.651138 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":254,"threads_started":1}
I20260809 23:02:21.100864 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:21.100039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:34621)
0809 23:02:21.100549 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:21.100645 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":238,"threads_started":1}
I20260809 23:02:21.569451 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:21.568453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:59747)
0809 23:02:21.569057 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:21.569240 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":300,"threads_started":1}
I20260809 23:02:21.652909 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:21.652213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:40387)
0809 23:02:21.652663 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:21.652743 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
I20260809 23:02:22.102557 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:22.101871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:49983)
0809 23:02:22.102283 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:22.102375 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":211,"threads_started":1}
I20260809 23:02:22.571199 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:22.570470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:55091)
0809 23:02:22.570917 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:22.571014 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":234,"threads_started":1}
I20260809 23:02:22.654737 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:22.654001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:50887)
0809 23:02:22.654467 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:22.654560 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":243,"threads_started":1}
I20260809 23:02:23.104323 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:23.103510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:47553)
0809 23:02:23.104077 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:23.104166 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":285,"threads_started":1}
I20260809 23:02:23.572834 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:23.572121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:44193)
0809 23:02:23.572592 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:23.572682 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":220,"threads_started":1}
I20260809 23:02:23.596230 10003 raft_consensus.cc:482] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:23.596462 10003 raft_consensus.cc:504] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:23.597203 10003 leader_election.cc:288] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409), 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789)
W20260809 23:02:23.598117 9530 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
W20260809 23:02:23.598333 9532 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
I20260809 23:02:23.598474 9532 leader_election.cc:302] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9fdbedf577c4f57acd051353e3383c5; no voters: 81c56cc956994e43b7b56f91f659c46f, a3b583f140284b668f7d2faab8b1c954
I20260809 23:02:23.598747 10003 raft_consensus.cc:2726] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:02:23.656844 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:23.655896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:48609)
0809 23:02:23.656472 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:23.656584 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":311,"threads_started":1}
I20260809 23:02:24.106019 10007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:24.105237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:53921)
0809 23:02:24.105721 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:24.105842 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":266,"threads_started":1}
I20260809 23:02:24.574860 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:24.573980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:53659)
0809 23:02:24.574605 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:24.574686 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":248,"threads_started":1}
I20260809 23:02:24.658500 10009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:24.657767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:51281)
0809 23:02:24.658218 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:24.658317 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":240,"threads_started":1}
I20260809 23:02:25.107723 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:25.107011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:38421)
0809 23:02:25.107469 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:25.107552 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":229,"threads_started":1}
I20260809 23:02:25.350499 10011 raft_consensus.cc:482] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:25.350728 10011 raft_consensus.cc:504] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:25.351420 10011 leader_election.cc:288] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409), 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789)
W20260809 23:02:25.352464 9530 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3b583f140284b668f7d2faab8b1c954 (127.24.31.194:41409): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
W20260809 23:02:25.352705 9532 leader_election.cc:341] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81c56cc956994e43b7b56f91f659c46f (127.24.31.195:38789): Not found: Tablet not found: 871a6d2a0b0a4857aa396b28a362eb54
I20260809 23:02:25.352844 9532 leader_election.cc:302] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9fdbedf577c4f57acd051353e3383c5; no voters: 81c56cc956994e43b7b56f91f659c46f, a3b583f140284b668f7d2faab8b1c954
I20260809 23:02:25.353210 10011 raft_consensus.cc:2726] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:02:25.576502 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:25.575809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:39189)
0809 23:02:25.576256 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:25.576337 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":222,"threads_started":1}
I20260809 23:02:25.660245 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:25.659461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:42315)
0809 23:02:25.659929 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:25.660031 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":238,"threads_started":1}
I20260809 23:02:26.109468 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:26.108686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:53899)
0809 23:02:26.109200 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:26.109280 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":291,"threads_started":1}
I20260809 23:02:26.578154 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:26.577488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:44505)
0809 23:02:26.577908 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:26.577990 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":231,"threads_started":1}
I20260809 23:02:26.662047 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:26.661215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:33799)
0809 23:02:26.661662 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:26.661840 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":228,"threads_started":1}
I20260809 23:02:27.111220 10018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:27.110430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:48753)
0809 23:02:27.110942 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:27.111043 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":244,"threads_started":1}
I20260809 23:02:27.579808 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:27.579032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:34817)
0809 23:02:27.579561 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:27.579649 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":310,"threads_started":1}
I20260809 23:02:27.663767 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:27.662949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:36403)
0809 23:02:27.663505 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:27.663585 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":330,"threads_started":1}
I20260809 23:02:28.112932 10021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:28.112122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:55903)
0809 23:02:28.112611 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:28.112719 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":239,"threads_started":1}
I20260809 23:02:28.581597 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:28.580775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:40487)
0809 23:02:28.581285 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:28.581398 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":227,"threads_started":1}
I20260809 23:02:28.665740 10023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:28.664947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.193:44993)
0809 23:02:28.665456 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:28.665557 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":294,"threads_started":1}
I20260809 23:02:29.115104 10024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:29.114231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.194:36717)
0809 23:02:29.114790 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:29.114927 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":260,"threads_started":1}
I20260809 23:02:29.564358 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 9467
I20260809 23:02:29.583391 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:29.582648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39623 (local address 127.24.31.195:34407)
0809 23:02:29.583144 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:29.583242 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:39623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":266,"threads_started":1}
I20260809 23:02:29.583837 10025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:29.565783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35980 (local address 127.24.31.193:38061)
0809 23:02:29.567114 (+ 1331us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:29.567127 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:29.567307 (+ 180us) server_negotiation.cc:407] Connection header received
0809 23:02:29.567495 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:29.567520 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:29.567837 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:29.567988 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:29.569438 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:29.570418 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:29.571702 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:29.572251 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:29.572349 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:29.578177 (+ 5828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:29.578189 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:29.578202 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:29.578241 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:29.580864 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:29.581617 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:29.581625 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:29.581627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:29.581764 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:29.582146 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:29.582150 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:29.582152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:29.582517 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:29.582851 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:29.583136 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 23:02:29.583441 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":967,"threads_started":1}
I20260809 23:02:29.585253 9651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:29.672588 10028 raft_consensus.cc:482] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:29.673061 10028 raft_consensus.cc:504] T 871a6d2a0b0a4857aa396b28a362eb54 P d9fdbedf577c4f57acd051353e3383c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3b583f140284b668f7d2faab8b1c954" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 41409 } } peers { permanent_uuid: "d9fdbedf577c4f57acd051353e3383c5" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 38061 } } peers { permanent_uuid: "81c56cc956994e43b7b56f91f659c46f" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 38789 } }
I20260809 23:02:29.673935 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 9467
I20260809 23:02:29.695147 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 9660
I20260809 23:02:29.708098 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:29.695911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.24.31.194:41409)
0809 23:02:29.696342 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:29.696346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:29.696363 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:02:29.696419 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:29.696422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:29.696482 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:29.696572 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:29.697982 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:29.698715 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:29.699948 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:29.700371 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:29.700463 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:29.703249 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:29.703254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:29.703258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:29.703290 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:29.705800 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:29.706484 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:29.706488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:29.706490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:29.706618 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:29.706962 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:29.706965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:29.706967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:29.707215 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:29.707407 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:29.707628 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 23:02:29.707792 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":209,"threads_started":1}
I20260809 23:02:29.708771 9806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:29.799064 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 9660
I20260809 23:02:29.819068 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 9814
I20260809 23:02:29.832986 10036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:29.819969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.24.31.195:38789)
0809 23:02:29.820541 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:29.820546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:29.820562 (+ 16us) server_negotiation.cc:407] Connection header received
0809 23:02:29.820614 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:29.820618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:29.820685 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:29.820804 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:29.822410 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:29.823172 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:29.824383 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:29.824837 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:29.824973 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:29.827668 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:29.827673 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:29.827676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:29.827707 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:29.830296 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:29.831000 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:29.831006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:29.831009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:29.831151 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:29.831523 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:29.831529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:29.831532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:29.831811 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:29.832078 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:29.832377 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 23:02:29.832612 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":272,"threads_started":1}
I20260809 23:02:29.833933 9955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:29.917805 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 9814
2026-08-09T23:02:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24696 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:02:30 dist-test-slave-vxj2 krb5kdc[10041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:02:30 dist-test-slave-vxj2 krb5kdc[10041](info): set up 2 sockets
Aug 09 23:02:30 dist-test-slave-vxj2 krb5kdc[10041](info): commencing operation
krb5kdc: starting...
W20260809 23:02:31.093724 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:02:31 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:02:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:02:31.368711 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44321
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:44321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:31.701745 10056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:31.702078 10056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:31.702230 10056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:31.709538 10056 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 23:02:31.709657 10056 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 23:02:31.709743 10056 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 23:02:31.709842 10056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:02:31.709939 10056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:31.710054 10056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:31.710135 10056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:02:31.710216 10056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:02:31.720023 10056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:44321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:31.721318 10056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:31.722652 10056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:31.734423 10062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:31.734408 10063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:31.734429 10065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:31.735308 10056 server_base.cc:1034] running on GCE node
I20260809 23:02:31.736106 10056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:31.737475 10056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:31.738718 10056 hybrid_clock.cc:648] HybridClock initialized: now 1786316551738641 us; error 45 us; skew 500 ppm
I20260809 23:02:31.739040 10056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:31 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316551, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:31.742198 10056 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:02:31.743543 10056 webserver.cc:460] Webserver started at http://127.24.31.254:43085/ using document root <none> and password file <none>
I20260809 23:02:31.744159 10056 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:31.744246 10056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:31.744516 10056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:31.747473 10056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "8b63c01156b04ad0b7caeef1544dabbf"
format_stamp: "Formatted at 2026-08-09 23:02:31 on dist-test-slave-vxj2"
server_key: "870d2a9699c061b0bbdd71ca3dd29a98"
server_key_iv: "4629e8483d6ca957ea169eb3a49c4484"
server_key_version: "encryptionkey@0"
I20260809 23:02:31.748189 10056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "8b63c01156b04ad0b7caeef1544dabbf"
format_stamp: "Formatted at 2026-08-09 23:02:31 on dist-test-slave-vxj2"
server_key: "870d2a9699c061b0bbdd71ca3dd29a98"
server_key_iv: "4629e8483d6ca957ea169eb3a49c4484"
server_key_version: "encryptionkey@0"
I20260809 23:02:31.753137 10056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 23:02:31.756479 10075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:31.757786 10056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 23:02:31.757954 10056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "8b63c01156b04ad0b7caeef1544dabbf"
format_stamp: "Formatted at 2026-08-09 23:02:31 on dist-test-slave-vxj2"
server_key: "870d2a9699c061b0bbdd71ca3dd29a98"
server_key_iv: "4629e8483d6ca957ea169eb3a49c4484"
server_key_version: "encryptionkey@0"
I20260809 23:02:31.758167 10056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:31.779621 10056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:31.783656 10056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:31.783929 10056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:31.802510 10056 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:44321
I20260809 23:02:31.802532 10140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:44321 every 8 connection(s)
I20260809 23:02:31.803929 10056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:02:31.807277 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 10056
I20260809 23:02:31.807530 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:02:31.808413 10141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:31.808470 24703 external_mini_cluster.cc:1428] Setting key ad2700bcb3ea4b9a91f75be017f8b0b2
I20260809 23:02:31.823940 10141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Bootstrap starting.
Aug 09 23:02:31 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:31.827548 10141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:31.828634 10141 log.cc:824] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:31.831485 10141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: No bootstrap required, opened a new log
I20260809 23:02:31.832844 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:31.810615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.24.31.254:44321)
0809 23:02:31.811414 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:31.811428 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:31.811476 (+ 48us) server_negotiation.cc:407] Connection header received
0809 23:02:31.813397 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:31.813420 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:31.813841 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:31.814177 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:31.816125 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:31.817528 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:31.819476 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:31.820189 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:31.820346 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:31.825505 (+ 5159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:31.825514 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:31.825529 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:31.825574 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:31.828682 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:31.829466 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:31.829473 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:31.829479 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:31.829640 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:31.830043 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:31.830049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:31.830052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:31.830560 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:31.830883 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:31.831319 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 23:02:31.831912 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":356,"threads_started":1}
I20260809 23:02:31.839923 10141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:02:31.840350 10141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:31.840485 10141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b63c01156b04ad0b7caeef1544dabbf, State: Initialized, Role: FOLLOWER
I20260809 23:02:31.841071 10141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:02:31.841383 10141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:02:31.841531 10141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:02:31.841732 10141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:31.844120 10141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:02:31.844612 10141 leader_election.cc:302] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b63c01156b04ad0b7caeef1544dabbf; no voters:
I20260809 23:02:31.845240 10141 leader_election.cc:288] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:02:31.845413 10146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:02:31.846102 10146 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 1 LEADER]: Becoming Leader. State: Replica: 8b63c01156b04ad0b7caeef1544dabbf, State: Running, Role: LEADER
I20260809 23:02:31.846727 10146 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:02:31.847520 10141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:02:31.849709 10148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b63c01156b04ad0b7caeef1544dabbf. Latest consensus state: current_term: 1 leader_uuid: "8b63c01156b04ad0b7caeef1544dabbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } } }
I20260809 23:02:31.850170 10148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:31.849702 10147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b63c01156b04ad0b7caeef1544dabbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } } }
I20260809 23:02:31.850907 10147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:31.853724 10154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:02:31.858623 10154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:02:31.867636 10154 catalog_manager.cc:1347] Generated new cluster ID: d183f5e24a754cde910165a9df860b4c
I20260809 23:02:31.867767 10154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:02:31.874689 10154 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:02:31.876025 10154 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:02:31.890247 10154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Generated new TSK 0
I20260809 23:02:31.891088 10154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 23:02:31.965063 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44321
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:32.301961 10169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:32.302235 10169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:32.302398 10169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:32.309976 10169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:32.310089 10169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:32.310277 10169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:02:32.320016 10169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:32.321399 10169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:32.322726 10169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:32.336480 10179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:32.336563 10175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:32.336510 10177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:32.337119 10169 server_base.cc:1034] running on GCE node
I20260809 23:02:32.337836 10169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:32.338703 10169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:32.339912 10169 hybrid_clock.cc:648] HybridClock initialized: now 1786316552339863 us; error 43 us; skew 500 ppm
I20260809 23:02:32.340229 10169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:32 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:32.343495 10169 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:02:32.345054 10169 webserver.cc:460] Webserver started at http://127.24.31.193:34369/ using document root <none> and password file <none>
I20260809 23:02:32.345803 10169 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:32.345947 10169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:32.346264 10169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:32.349442 10169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "e43fc3fedbf14297bc8a264650d5344a"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "ad2491ce1b50b415648e9138ef5a1c48"
server_key_iv: "062a53251c3f60a36bea0ebe9f9287a3"
server_key_version: "encryptionkey@0"
I20260809 23:02:32.350241 10169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "e43fc3fedbf14297bc8a264650d5344a"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "ad2491ce1b50b415648e9138ef5a1c48"
server_key_iv: "062a53251c3f60a36bea0ebe9f9287a3"
server_key_version: "encryptionkey@0"
I20260809 23:02:32.355724 10169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:02:32.359202 10186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:32.360512 10169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:02:32.360678 10169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "e43fc3fedbf14297bc8a264650d5344a"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "ad2491ce1b50b415648e9138ef5a1c48"
server_key_iv: "062a53251c3f60a36bea0ebe9f9287a3"
server_key_version: "encryptionkey@0"
I20260809 23:02:32.360893 10169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:32.387643 10169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:32.392261 10169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:32.392573 10169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:32.393568 10169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:32.394944 10169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:32.395048 10169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:32.395201 10169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:32.395272 10169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:32.428934 10169 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:37095
I20260809 23:02:32.428984 10315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:37095 every 8 connection(s)
I20260809 23:02:32.430401 10169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:02:32.431551 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 10169
I20260809 23:02:32.431790 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:02:32.432528 24703 external_mini_cluster.cc:1428] Setting key 870ebbe4317a9e3f4ea4bb12c5703662
Aug 09 23:02:32 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:32.456902 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:32.433213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:49879 (local address 127.24.31.254:44321)
0809 23:02:32.433778 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:32.433785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:32.437519 (+ 3734us) server_negotiation.cc:407] Connection header received
0809 23:02:32.440660 (+ 3141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:32.440664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:32.440737 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:32.440835 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:32.443228 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:32.443964 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:32.446224 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:32.446875 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:32.447019 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:32.451016 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:32.451021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:32.451025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:32.451058 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:32.453737 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:32.454547 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:32.454552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:32.454554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:32.454674 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:32.455105 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:32.455110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:32.455114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:32.455434 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:32.455592 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:32.456293 (+ 701us) server_negotiation.cc:299] Negotiation successful
0809 23:02:32.456525 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":265,"threads_started":1}
I20260809 23:02:32.457911 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:32.433931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:49879)
0809 23:02:32.437331 (+ 3400us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:32.437384 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:32.440258 (+ 2874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:32.441130 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:32.441145 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:32.442050 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:32.442915 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:32.442945 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:32.444149 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:32.444158 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:32.446058 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:32.446071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:32.447133 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:32.447141 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:32.447378 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:32.448182 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:32.448204 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:32.450769 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:32.453878 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:32.453887 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:32.453903 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:32.454408 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:32.454785 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:32.454790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:32.454794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:32.454986 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:32.455571 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:32.455579 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:32.455903 (+ 324us) client_negotiation.cc:769] Sending connection context
0809 23:02:32.456173 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 23:02:32.456727 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":378,"threads_started":1}
I20260809 23:02:32.459609 10316 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44321
I20260809 23:02:32.460000 10316 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:32.461112 10316 heartbeater.cc:511] Master 127.24.31.254:44321 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
I20260809 23:02:32.463759 10095 ts_manager.cc:194] Registered new tserver with Master: e43fc3fedbf14297bc8a264650d5344a (127.24.31.193:37095)
I20260809 23:02:32.465425 10095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:49879
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 23:02:32.495779 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44321
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:32.832379 10322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:32.832638 10322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:32.832731 10322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:32.839844 10322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:32.839979 10322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:32.840130 10322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:02:32.849696 10322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:32.850947 10322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:32.852159 10322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:32.865600 10331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:32.865633 10328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:32.865568 10329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:32.866068 10322 server_base.cc:1034] running on GCE node
I20260809 23:02:32.867053 10322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:32.867879 10322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:32.869112 10322 hybrid_clock.cc:648] HybridClock initialized: now 1786316552869081 us; error 41 us; skew 500 ppm
I20260809 23:02:32.869432 10322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:32 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:32.872426 10322 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:02:32.873845 10322 webserver.cc:460] Webserver started at http://127.24.31.194:36979/ using document root <none> and password file <none>
I20260809 23:02:32.874519 10322 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:32.874670 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:32.874965 10322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:32.877858 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "90ec54e6dc8a4ead821d9eb5acd8950f"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "f56f100742d7a77e6d84ea0ff7a7dbd3"
server_key_iv: "7925ace75247ce55263729cd31dd45cf"
server_key_version: "encryptionkey@0"
I20260809 23:02:32.878635 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "90ec54e6dc8a4ead821d9eb5acd8950f"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "f56f100742d7a77e6d84ea0ff7a7dbd3"
server_key_iv: "7925ace75247ce55263729cd31dd45cf"
server_key_version: "encryptionkey@0"
I20260809 23:02:32.883503 10322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 23:02:32.886791 10340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:32.887959 10322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 23:02:32.888125 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "90ec54e6dc8a4ead821d9eb5acd8950f"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "f56f100742d7a77e6d84ea0ff7a7dbd3"
server_key_iv: "7925ace75247ce55263729cd31dd45cf"
server_key_version: "encryptionkey@0"
I20260809 23:02:32.888315 10322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:32.905450 10322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:32.909521 10322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:32.909809 10322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:32.910806 10322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:32.912137 10322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:32.912240 10322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:32.912380 10322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:32.912456 10322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:32.947664 10322 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:38193
I20260809 23:02:32.947752 10471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:38193 every 8 connection(s)
I20260809 23:02:32.949113 10322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:02:32.952157 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 10322
I20260809 23:02:32.952358 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:02:32.952848 24703 external_mini_cluster.cc:1428] Setting key df453a2d68fd8d5447aec025dd8df1f9
Aug 09 23:02:32 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316552, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:32.970669 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:32.951600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:59709 (local address 127.24.31.254:44321)
0809 23:02:32.951835 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:32.951842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:32.953097 (+ 1255us) server_negotiation.cc:407] Connection header received
0809 23:02:32.955298 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:32.955303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:32.955394 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:32.955521 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:32.957007 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:32.958176 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:32.959855 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:32.960614 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:32.960752 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:32.964487 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:32.964495 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:32.964503 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:32.964548 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:32.967334 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:32.968273 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:32.968277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:32.968279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:32.968402 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:32.968867 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:32.968870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:32.968872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:32.969207 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:32.969348 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:32.970265 (+ 917us) server_negotiation.cc:299] Negotiation successful
0809 23:02:32.970435 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:02:32.972185 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:32.952083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:59709)
0809 23:02:32.952895 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:32.952961 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:32.955047 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:32.955702 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:32.955729 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:32.956284 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:32.956833 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:32.956851 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:32.958320 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:32.958325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:32.959705 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:32.959718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:32.960851 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:32.960859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:32.961199 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:32.961982 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:32.962009 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:32.964266 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:32.967466 (+ 3200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:32.967477 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:32.967494 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:32.968137 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:32.968508 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:32.968516 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:32.968522 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:32.968760 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:32.969357 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:32.969368 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:32.969842 (+ 474us) client_negotiation.cc:769] Sending connection context
0809 23:02:32.970181 (+ 339us) client_negotiation.cc:240] Negotiation successful
0809 23:02:32.970716 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":353,"threads_started":1}
I20260809 23:02:32.973605 10473 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44321
I20260809 23:02:32.973938 10473 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:32.974647 10473 heartbeater.cc:511] Master 127.24.31.254:44321 requested a full tablet report, sending...
I20260809 23:02:32.976187 10095 ts_manager.cc:194] Registered new tserver with Master: 90ec54e6dc8a4ead821d9eb5acd8950f (127.24.31.194:38193)
I20260809 23:02:32.977041 10095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:59709
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 23:02:33.014833 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44321
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:33.357425 10479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:33.357682 10479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:33.357852 10479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:33.365137 10479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:33.365278 10479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:33.365494 10479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:02:33.374961 10479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:33.376235 10479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:33.377624 10479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:33.392242 10485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:33.392324 10487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:33.392665 10484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:33.393177 10479 server_base.cc:1034] running on GCE node
I20260809 23:02:33.393803 10479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:33.394696 10479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:33.395893 10479 hybrid_clock.cc:648] HybridClock initialized: now 1786316553395851 us; error 31 us; skew 500 ppm
I20260809 23:02:33.396277 10479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:33 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316553, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:33.399432 10479 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:02:33.400771 10479 webserver.cc:460] Webserver started at http://127.24.31.195:46669/ using document root <none> and password file <none>
I20260809 23:02:33.401491 10479 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:33.401639 10479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:33.401928 10479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:02:33.404857 10479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "a3f80c45f7bb41bb808addac45ad4592"
format_stamp: "Formatted at 2026-08-09 23:02:33 on dist-test-slave-vxj2"
server_key: "8b3292723c1de0c34a8d3e78c0311b14"
server_key_iv: "4e487772d50c513e748f6410feb3fd48"
server_key_version: "encryptionkey@0"
I20260809 23:02:33.405597 10479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "a3f80c45f7bb41bb808addac45ad4592"
format_stamp: "Formatted at 2026-08-09 23:02:33 on dist-test-slave-vxj2"
server_key: "8b3292723c1de0c34a8d3e78c0311b14"
server_key_iv: "4e487772d50c513e748f6410feb3fd48"
server_key_version: "encryptionkey@0"
I20260809 23:02:33.410594 10479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:02:33.413890 10495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:33.415057 10479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260809 23:02:33.415220 10479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "a3f80c45f7bb41bb808addac45ad4592"
format_stamp: "Formatted at 2026-08-09 23:02:33 on dist-test-slave-vxj2"
server_key: "8b3292723c1de0c34a8d3e78c0311b14"
server_key_iv: "4e487772d50c513e748f6410feb3fd48"
server_key_version: "encryptionkey@0"
I20260809 23:02:33.415426 10479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:33.439893 10479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:33.444254 10479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:33.444566 10479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:33.445600 10479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:33.446945 10479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:33.447019 10479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:33.447178 10479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:33.447230 10479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:33.468370 10316 heartbeater.cc:503] Master 127.24.31.254:44321 was elected leader, sending a full tablet report...
I20260809 23:02:33.480343 10479 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:34761
I20260809 23:02:33.480403 10625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:34761 every 8 connection(s)
I20260809 23:02:33.481742 10479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:02:33.490988 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 10479
I20260809 23:02:33.491197 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:02:33.491753 24703 external_mini_cluster.cc:1428] Setting key a118b8581637cae960a71452ea1b313e
Aug 09 23:02:33 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316553, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:33.502810 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.484289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:49389 (local address 127.24.31.254:44321)
0809 23:02:33.484792 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:33.484798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:33.486087 (+ 1289us) server_negotiation.cc:407] Connection header received
0809 23:02:33.487785 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:33.487791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:33.487885 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:33.487998 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:33.489506 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.490320 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.492642 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.493253 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.493369 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.497130 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:33.497135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:33.497138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:33.497208 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:33.500133 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:33.500803 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:33.500807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:33.500809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:33.500941 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:33.501366 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:33.501369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:33.501371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:33.501643 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:33.501755 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:33.502384 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 23:02:33.502549 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":246,"threads_started":1}
I20260809 23:02:33.503772 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.484932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:49389)
0809 23:02:33.485823 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:33.485866 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:33.487537 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:33.488199 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:33.488207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:33.488714 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:33.489335 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.489353 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.490450 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.490453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.492448 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.492461 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.493494 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.493502 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.493803 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.494543 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:33.494564 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:33.496949 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:33.500254 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:33.500261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:33.500273 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:33.500706 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:33.501030 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:33.501034 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:33.501036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:33.501273 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:33.501752 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:33.501757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:33.502059 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 23:02:33.502292 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 23:02:33.502678 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":461,"threads_started":1}
I20260809 23:02:33.505084 10626 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44321
I20260809 23:02:33.505420 10626 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:33.506321 10626 heartbeater.cc:511] Master 127.24.31.254:44321 requested a full tablet report, sending...
I20260809 23:02:33.507961 10095 ts_manager.cc:194] Registered new tserver with Master: a3f80c45f7bb41bb808addac45ad4592 (127.24.31.195:34761)
I20260809 23:02:33.508703 10095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:49389
I20260809 23:02:33.519688 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:02:33.536652 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.524210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44386 (local address 127.24.31.254:44321)
0809 23:02:33.524451 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:33.524461 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:33.524763 (+ 302us) server_negotiation.cc:407] Connection header received
0809 23:02:33.524912 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:33.524916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:33.524981 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:33.525058 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:33.526531 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.527261 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.528688 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.529096 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.529257 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.531681 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:33.531686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:33.531706 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:33.531734 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:33.534320 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:33.534958 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:33.534962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:33.534964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:33.535078 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:33.535390 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:33.535393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:33.535395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:33.535708 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:33.535884 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:33.536150 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 23:02:33.536318 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 23:02:33.540805 10095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:02:33.547923 10095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:02:33.583246 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.561450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34410 (local address 127.24.31.193:37095)
0809 23:02:33.573284 (+ 11834us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:33.573295 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:33.573329 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:02:33.573434 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:33.573442 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:33.573713 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:33.573921 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:33.575124 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.576809 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.580526 (+ 3717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.581991 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.582148 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.582408 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:33.582724 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 23:02:33.582900 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11637,"thread_start_us":11489,"threads_started":1}
I20260809 23:02:33.583318 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.560683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37095 (local address 127.0.0.1:34410)
0809 23:02:33.572511 (+ 11828us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:33.572533 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:33.572669 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:33.574368 (+ 1699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:33.574376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:33.574406 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:33.574898 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.574914 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.576982 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.576987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.580339 (+ 3352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.580355 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.582206 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.582212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.582452 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.582516 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 23:02:33.582676 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 23:02:33.582880 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11666,"mutex_wait_us":214,"thread_start_us":16553,"threads_started":1}
I20260809 23:02:33.584162 10642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.560593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45802 (local address 127.24.31.194:38193)
0809 23:02:33.573656 (+ 13063us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:33.573664 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:33.573690 (+ 26us) server_negotiation.cc:407] Connection header received
0809 23:02:33.573781 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:33.573787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:33.574001 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:33.574197 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:33.575242 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.576802 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.580761 (+ 3959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.582288 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.582906 (+ 618us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.583105 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:33.583677 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 23:02:33.583850 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12905,"thread_start_us":13525,"threads_started":1}
I20260809 23:02:33.585027 10646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.560571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:38193 (local address 127.0.0.1:45802)
0809 23:02:33.570712 (+ 10141us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:33.570774 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:33.570999 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:33.574547 (+ 3548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:33.574553 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:33.574579 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:33.575045 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.575055 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.576982 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.576987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.580339 (+ 3352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.580355 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.583268 (+ 2913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.583275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.583461 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.583495 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 23:02:33.584387 (+ 892us) client_negotiation.cc:240] Negotiation successful
0809 23:02:33.584550 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9823,"mutex_wait_us":267,"thread_start_us":11636,"threads_started":1}
I20260809 23:02:33.585734 10645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.561123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:34761 (local address 127.0.0.1:32846)
0809 23:02:33.577612 (+ 16489us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:33.577628 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:33.577779 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:33.578630 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:33.578634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:33.578658 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:33.579159 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.579168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.580573 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.580578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.582771 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.582786 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.584917 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.584930 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.585169 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.585205 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 23:02:33.585302 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 23:02:33.585432 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16344,"mutex_wait_us":20,"thread_start_us":9090,"threads_started":1}
I20260809 23:02:33.586323 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.561801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.24.31.195:34761)
0809 23:02:33.576742 (+ 14941us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:33.576754 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:33.577910 (+ 1156us) server_negotiation.cc:407] Connection header received
0809 23:02:33.578019 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:33.578027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:33.578246 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:33.578422 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:33.579327 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.580360 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.582973 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.584407 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.584525 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.584763 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:33.585968 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0809 23:02:33.586087 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14708,"thread_start_us":14942,"threads_started":1}
I20260809 23:02:33.588339 10241 tablet_service.cc:1467] Processing CreateTablet for tablet c9b223992b724c6aa54804c6b75233a2 (DEFAULT_TABLE table=test-table [id=f9824e97ef3f4d069f7a28254e75cdd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:33.589479 10551 tablet_service.cc:1467] Processing CreateTablet for tablet c9b223992b724c6aa54804c6b75233a2 (DEFAULT_TABLE table=test-table [id=f9824e97ef3f4d069f7a28254e75cdd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:33.590018 10241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b223992b724c6aa54804c6b75233a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:33.589919 10395 tablet_service.cc:1467] Processing CreateTablet for tablet c9b223992b724c6aa54804c6b75233a2 (DEFAULT_TABLE table=test-table [id=f9824e97ef3f4d069f7a28254e75cdd8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:33.590786 10551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b223992b724c6aa54804c6b75233a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:33.591190 10395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9b223992b724c6aa54804c6b75233a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:33.602432 10649 tablet_bootstrap.cc:492] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: Bootstrap starting.
I20260809 23:02:33.605572 10651 tablet_bootstrap.cc:492] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: Bootstrap starting.
I20260809 23:02:33.605890 10649 tablet_bootstrap.cc:654] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:33.607590 10649 log.cc:824] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:33.609597 10651 tablet_bootstrap.cc:654] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:33.610229 10650 tablet_bootstrap.cc:492] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: Bootstrap starting.
I20260809 23:02:33.611018 10651 log.cc:824] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:33.612394 10649 tablet_bootstrap.cc:492] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: No bootstrap required, opened a new log
I20260809 23:02:33.612685 10649 ts_tablet_manager.cc:1397] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 23:02:33.613747 10651 tablet_bootstrap.cc:492] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: No bootstrap required, opened a new log
I20260809 23:02:33.614017 10651 ts_tablet_manager.cc:1397] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 23:02:33.615326 10650 tablet_bootstrap.cc:654] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:33.616626 10650 log.cc:824] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:33.619848 10650 tablet_bootstrap.cc:492] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: No bootstrap required, opened a new log
I20260809 23:02:33.620146 10650 ts_tablet_manager.cc:1397] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260809 23:02:33.622392 10651 raft_consensus.cc:348] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } }
I20260809 23:02:33.622753 10651 raft_consensus.cc:374] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:33.622854 10651 raft_consensus.cc:729] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ec54e6dc8a4ead821d9eb5acd8950f, State: Initialized, Role: FOLLOWER
I20260809 23:02:33.623430 10651 consensus_queue.cc:260] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } }
I20260809 23:02:33.623318 10649 raft_consensus.cc:348] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } }
I20260809 23:02:33.623723 10649 raft_consensus.cc:374] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:33.623821 10649 raft_consensus.cc:729] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f80c45f7bb41bb808addac45ad4592, State: Initialized, Role: FOLLOWER
I20260809 23:02:33.624402 10649 consensus_queue.cc:260] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } }
I20260809 23:02:33.625679 10626 heartbeater.cc:503] Master 127.24.31.254:44321 was elected leader, sending a full tablet report...
I20260809 23:02:33.626256 10649 ts_tablet_manager.cc:1428] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260809 23:02:33.627218 10473 heartbeater.cc:503] Master 127.24.31.254:44321 was elected leader, sending a full tablet report...
I20260809 23:02:33.627806 10651 ts_tablet_manager.cc:1428] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260809 23:02:33.633033 10650 raft_consensus.cc:348] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } }
I20260809 23:02:33.633432 10650 raft_consensus.cc:374] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:33.633545 10650 raft_consensus.cc:729] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e43fc3fedbf14297bc8a264650d5344a, State: Initialized, Role: FOLLOWER
I20260809 23:02:33.634038 10650 consensus_queue.cc:260] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } }
I20260809 23:02:33.635566 10650 ts_tablet_manager.cc:1428] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
W20260809 23:02:33.682937 10317 tablet.cc:2369] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:02:33.701128 10474 tablet.cc:2369] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:02:33.733327 10627 tablet.cc:2369] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:02:33.907749 10656 raft_consensus.cc:482] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:33.908092 10656 raft_consensus.cc:504] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } }
I20260809 23:02:33.909986 10656 leader_election.cc:288] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90ec54e6dc8a4ead821d9eb5acd8950f (127.24.31.194:38193), e43fc3fedbf14297bc8a264650d5344a (127.24.31.193:37095)
I20260809 23:02:33.917790 10642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.910652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:42249 (local address 127.24.31.194:38193)
0809 23:02:33.910922 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:33.910928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:33.910968 (+ 40us) server_negotiation.cc:407] Connection header received
0809 23:02:33.911082 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:33.911087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:33.911172 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:33.911263 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:33.911712 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.912759 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.915704 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.916846 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.917024 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.917102 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:33.917431 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 23:02:33.917564 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 23:02:33.917948 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.910343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:38193 (local address 127.24.31.195:42249)
0809 23:02:33.910744 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:33.910764 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:33.910917 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:33.911324 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:33.911331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:33.911370 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:33.911577 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.911585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.912922 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.912926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.915535 (+ 2609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.915553 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.917024 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.917029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.917258 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.917289 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 23:02:33.917382 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 23:02:33.917574 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":34}
I20260809 23:02:33.918462 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.910526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:58165 (local address 127.24.31.193:37095)
0809 23:02:33.910748 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:33.910756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:33.911389 (+ 633us) server_negotiation.cc:407] Connection header received
0809 23:02:33.911532 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:33.911536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:33.911592 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:33.911695 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:33.912259 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.913224 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.916085 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.917332 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.917516 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.917582 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:33.918038 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 23:02:33.918206 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 23:02:33.919080 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.910396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37095 (local address 127.24.31.195:58165)
0809 23:02:33.911272 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:33.911289 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:33.911397 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:33.911725 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:33.911728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:33.911746 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:33.912101 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.912110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.913408 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.913414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.915922 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:33.915935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.917566 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.917573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:33.917789 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.917828 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 23:02:33.918532 (+ 704us) client_negotiation.cc:240] Negotiation successful
0809 23:02:33.918689 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":51,"thread_start_us":506,"threads_started":1}
I20260809 23:02:33.919071 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b223992b724c6aa54804c6b75233a2" candidate_uuid: "a3f80c45f7bb41bb808addac45ad4592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" is_pre_election: true
I20260809 23:02:33.919668 10423 raft_consensus.cc:2454] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3f80c45f7bb41bb808addac45ad4592 in term 0.
I20260809 23:02:33.920360 10496 leader_election.cc:302] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ec54e6dc8a4ead821d9eb5acd8950f, a3f80c45f7bb41bb808addac45ad4592; no voters:
I20260809 23:02:33.920327 10264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b223992b724c6aa54804c6b75233a2" candidate_uuid: "a3f80c45f7bb41bb808addac45ad4592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e43fc3fedbf14297bc8a264650d5344a" is_pre_election: true
I20260809 23:02:33.920837 10656 raft_consensus.cc:2781] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:02:33.920918 10264 raft_consensus.cc:2454] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3f80c45f7bb41bb808addac45ad4592 in term 0.
I20260809 23:02:33.921029 10656 raft_consensus.cc:482] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:02:33.921188 10656 raft_consensus.cc:3037] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:33.923949 10656 raft_consensus.cc:504] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } }
I20260809 23:02:33.924681 10656 leader_election.cc:288] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [CANDIDATE]: Term 1 election: Requested vote from peers 90ec54e6dc8a4ead821d9eb5acd8950f (127.24.31.194:38193), e43fc3fedbf14297bc8a264650d5344a (127.24.31.193:37095)
I20260809 23:02:33.925115 10423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b223992b724c6aa54804c6b75233a2" candidate_uuid: "a3f80c45f7bb41bb808addac45ad4592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f"
I20260809 23:02:33.925292 10264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9b223992b724c6aa54804c6b75233a2" candidate_uuid: "a3f80c45f7bb41bb808addac45ad4592" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e43fc3fedbf14297bc8a264650d5344a"
I20260809 23:02:33.925421 10423 raft_consensus.cc:3037] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:33.925527 10264 raft_consensus.cc:3037] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:33.927764 10264 raft_consensus.cc:2454] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3f80c45f7bb41bb808addac45ad4592 in term 1.
I20260809 23:02:33.928056 10423 raft_consensus.cc:2454] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3f80c45f7bb41bb808addac45ad4592 in term 1.
I20260809 23:02:33.928174 10500 leader_election.cc:302] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3f80c45f7bb41bb808addac45ad4592, e43fc3fedbf14297bc8a264650d5344a; no voters:
I20260809 23:02:33.928496 10656 raft_consensus.cc:2781] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:02:33.928987 10656 raft_consensus.cc:686] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 1 LEADER]: Becoming Leader. State: Replica: a3f80c45f7bb41bb808addac45ad4592, State: Running, Role: LEADER
I20260809 23:02:33.929621 10656 consensus_queue.cc:237] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } }
I20260809 23:02:33.934773 10094 catalog_manager.cc:5495] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3f80c45f7bb41bb808addac45ad4592 (127.24.31.195). New cstate: current_term: 1 leader_uuid: "a3f80c45f7bb41bb808addac45ad4592" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } health_report { overall_health: HEALTHY } } }
I20260809 23:02:33.954747 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.948922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.24.31.195:34761)
0809 23:02:33.949147 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:33.949193 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:33.949383 (+ 190us) server_negotiation.cc:407] Connection header received
0809 23:02:33.949626 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:33.949631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:33.949715 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:33.949875 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:02:33.950303 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.951030 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.952636 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:33.953056 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:33.953210 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:33.953694 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:02:33.954161 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:02:33.954317 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:33.954399 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 23:02:33.954482 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:02:33.962612 10423 raft_consensus.cc:1261] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [term 1 FOLLOWER]: Refusing update from remote peer a3f80c45f7bb41bb808addac45ad4592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:02:33.962843 10264 raft_consensus.cc:1261] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [term 1 FOLLOWER]: Refusing update from remote peer a3f80c45f7bb41bb808addac45ad4592: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:02:33.963691 10656 consensus_queue.cc:1035] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:02:33.964020 10659 consensus_queue.cc:1035] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:02:33.989307 10095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:02:33.989602 10095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:02:33.992967 10095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Sending DeleteTablet for 3 replicas of tablet c9b223992b724c6aa54804c6b75233a2
I20260809 23:02:33.994252 10241 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b223992b724c6aa54804c6b75233a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:33 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:34410
I20260809 23:02:33.994333 10395 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b223992b724c6aa54804c6b75233a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:33 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:45802
I20260809 23:02:33.995193 10673 tablet_replica.cc:330] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: stopping tablet replica
I20260809 23:02:33.995690 10673 raft_consensus.cc:2229] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:02:33.996196 10673 raft_consensus.cc:2258] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:33.996382 10551 tablet_service.cc:1514] Processing DeleteTablet for tablet c9b223992b724c6aa54804c6b75233a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:33 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:32846
I20260809 23:02:33.996762 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 10169
I20260809 23:02:33.998811 10673 ts_tablet_manager.cc:1905] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:34.005755 10676 tablet_replica.cc:330] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: stopping tablet replica
I20260809 23:02:34.006283 10676 raft_consensus.cc:2229] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:02:34.007134 10676 raft_consensus.cc:2258] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:34.009521 10676 ts_tablet_manager.cc:1905] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:34.010277 10673 ts_tablet_manager.cc:1918] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:34.010484 10673 log.cc:1196] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/c9b223992b724c6aa54804c6b75233a2
I20260809 23:02:34.011173 10673 ts_tablet_manager.cc:1939] T c9b223992b724c6aa54804c6b75233a2 P e43fc3fedbf14297bc8a264650d5344a: Deleting consensus metadata
I20260809 23:02:34.013954 10079 catalog_manager.cc:4841] TS e43fc3fedbf14297bc8a264650d5344a (127.24.31.193:37095): tablet c9b223992b724c6aa54804c6b75233a2 (table test-table [id=f9824e97ef3f4d069f7a28254e75cdd8]) successfully deleted
I20260809 23:02:34.017534 10676 ts_tablet_manager.cc:1918] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:34.017729 10676 log.cc:1196] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/c9b223992b724c6aa54804c6b75233a2
I20260809 23:02:34.018362 10676 ts_tablet_manager.cc:1939] T c9b223992b724c6aa54804c6b75233a2 P a3f80c45f7bb41bb808addac45ad4592: Deleting consensus metadata
I20260809 23:02:34.019414 10674 tablet_replica.cc:330] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: stopping tablet replica
I20260809 23:02:34.020143 10674 raft_consensus.cc:2229] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:02:34.020336 10077 catalog_manager.cc:4841] TS a3f80c45f7bb41bb808addac45ad4592 (127.24.31.195:34761): tablet c9b223992b724c6aa54804c6b75233a2 (table test-table [id=f9824e97ef3f4d069f7a28254e75cdd8]) successfully deleted
I20260809 23:02:34.020803 10674 raft_consensus.cc:2258] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:34.023145 10674 ts_tablet_manager.cc:1905] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:34.031009 10674 ts_tablet_manager.cc:1918] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:34.031203 10674 log.cc:1196] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/c9b223992b724c6aa54804c6b75233a2
Aug 09 23:02:34 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 23:02:34.031807 10674 ts_tablet_manager.cc:1939] T c9b223992b724c6aa54804c6b75233a2 P 90ec54e6dc8a4ead821d9eb5acd8950f: Deleting consensus metadata
I20260809 23:02:34.033699 10076 catalog_manager.cc:4841] TS 90ec54e6dc8a4ead821d9eb5acd8950f (127.24.31.194:38193): tablet c9b223992b724c6aa54804c6b75233a2 (table test-table [id=f9824e97ef3f4d069f7a28254e75cdd8]) successfully deleted
I20260809 23:02:34.039683 10647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:33.998049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.24.31.193:37095)
0809 23:02:33.998219 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:33.998227 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:34.018303 (+ 20076us) server_negotiation.cc:407] Connection header received
0809 23:02:34.018695 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:34.018705 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:34.018814 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:34.018945 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:34.022403 (+ 3458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:34.023572 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:34.025886 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:34.026541 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:34.026674 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:34.032961 (+ 6287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:34.032980 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:34.032999 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:34.033048 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:34.036425 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:34.037327 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:34.037334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:34.037337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:34.037497 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:34.037965 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:34.037969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:34.037971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:34.038400 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:34.038658 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:34.039139 (+ 481us) server_negotiation.cc:299] Negotiation successful
0809 23:02:34.039302 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 23:02:34.040315 10311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:34.131373 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 10169
I20260809 23:02:34.153075 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 10322
Aug 09 23:02:34 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 23:02:34.169839 10642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:34.154007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45810 (local address 127.24.31.194:38193)
0809 23:02:34.154215 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:34.154225 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:34.154338 (+ 113us) server_negotiation.cc:407] Connection header received
0809 23:02:34.154547 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:34.154551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:34.154617 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:34.154721 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:34.156161 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:34.156886 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:34.158448 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:34.158860 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:34.158991 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:34.163977 (+ 4986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:34.163993 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:34.164009 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:34.164047 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:34.166859 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:34.167583 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:34.167590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:34.167592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:34.167725 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:34.168127 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:34.168132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:34.168135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:34.168515 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:34.168818 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:34.169142 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 23:02:34.169543 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 23:02:34.170624 10467 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:34.262143 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 10322
I20260809 23:02:34.282975 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 10479
Aug 09 23:02:34 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 23:02:34.301585 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:34.283865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.24.31.195:34761)
0809 23:02:34.284377 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:34.284382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:34.284399 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:02:34.284461 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:34.284465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:34.284524 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:34.284662 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:34.286299 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:34.287236 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:34.288677 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:34.289140 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:34.289498 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:34.294792 (+ 5294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:34.294805 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:34.294826 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:34.294880 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:34.298244 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:34.299038 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:34.299047 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:34.299049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:34.299264 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:34.299764 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:34.299778 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:34.299781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:34.300171 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:34.300574 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:34.300920 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 23:02:34.301182 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366}
I20260809 23:02:34.302464 10621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:34.427788 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 10479
I20260809 23:02:34.447028 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 10056
I20260809 23:02:34.448228 10136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:34.561348 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 10056
I20260809 23:02:54.586150 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44321
--webserver_interface=127.24.31.254
--webserver_port=43085
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:44321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:02:54.932004 10683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:54.932354 10683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:54.932540 10683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:54.939990 10683 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 23:02:54.940138 10683 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 23:02:54.940244 10683 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 23:02:54.940351 10683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:02:54.940466 10683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:54.940589 10683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:54.940689 10683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:02:54.940796 10683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:02:54.951067 10683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:44321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=43085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:54.952490 10683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:54.953928 10683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:54.978387 10693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:54.978396 10696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:54.979023 10691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:54.982724 10683 server_base.cc:1034] running on GCE node
I20260809 23:02:54.983711 10683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:54.985419 10683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:54.986714 10683 hybrid_clock.cc:648] HybridClock initialized: now 1786316574986627 us; error 53 us; skew 500 ppm
I20260809 23:02:54.987196 10683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:54 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316574, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:54.991178 10683 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:02:54.992928 10683 webserver.cc:460] Webserver started at http://127.24.31.254:43085/ using document root <none> and password file <none>
I20260809 23:02:54.993801 10683 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:54.993908 10683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:54.999868 10683 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 23:02:55.003144 10705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:55.004602 10683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:02:55.004843 10683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "8b63c01156b04ad0b7caeef1544dabbf"
format_stamp: "Formatted at 2026-08-09 23:02:31 on dist-test-slave-vxj2"
server_key: "870d2a9699c061b0bbdd71ca3dd29a98"
server_key_iv: "4629e8483d6ca957ea169eb3a49c4484"
server_key_version: "encryptionkey@0"
I20260809 23:02:55.006067 10683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:55.024075 10683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:55.028993 10683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:55.029325 10683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:55.048416 10683 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:44321
I20260809 23:02:55.048461 10769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:44321 every 8 connection(s)
I20260809 23:02:55.050086 10683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:02:55.054365 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 10683
I20260809 23:02:55.055460 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:37095
--local_ip_for_outbound_sockets=127.24.31.193
--tserver_master_addrs=127.24.31.254:44321
--webserver_port=34369
--webserver_interface=127.24.31.193
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:02:55.059490 10770 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:02:55.065256 10770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Bootstrap starting.
I20260809 23:02:55.087381 10770 log.cc:824] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:55.100405 10770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:02:55.101025 10770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Bootstrap complete.
I20260809 23:02:55.111542 10770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:02:55.112454 10770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b63c01156b04ad0b7caeef1544dabbf, State: Initialized, Role: FOLLOWER
I20260809 23:02:55.113296 10770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:02:55.113569 10770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:02:55.113706 10770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:02:55.113909 10770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:02:55.118201 10770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:02:55.118772 10770 leader_election.cc:302] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b63c01156b04ad0b7caeef1544dabbf; no voters:
I20260809 23:02:55.119457 10770 leader_election.cc:288] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:02:55.119616 10774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:02:55.120896 10774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 2 LEADER]: Becoming Leader. State: Replica: 8b63c01156b04ad0b7caeef1544dabbf, State: Running, Role: LEADER
I20260809 23:02:55.121735 10774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:02:55.122164 10770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:02:55.124681 10775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8b63c01156b04ad0b7caeef1544dabbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } } }
I20260809 23:02:55.125365 10776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b63c01156b04ad0b7caeef1544dabbf. Latest consensus state: current_term: 2 leader_uuid: "8b63c01156b04ad0b7caeef1544dabbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } } }
I20260809 23:02:55.125834 10775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:55.125938 10776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: This master's current role is: LEADER
I20260809 23:02:55.133920 10780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:02:55.144562 10780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:02:55.147061 10780 catalog_manager.cc:1259] Loaded cluster ID: d183f5e24a754cde910165a9df860b4c
I20260809 23:02:55.147297 10780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:02:55.153172 10780 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:02:55.155596 10780 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Loaded TSK: 0
I20260809 23:02:55.182741 10793 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 23:02:55.196141 10780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Generated new TSK 1
I20260809 23:02:55.196558 10780 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:02:55.439330 10772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:55.439643 10772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:55.439816 10772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:55.447788 10772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:55.447939 10772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:55.448137 10772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:02:55.458149 10772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:37095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=34369
--tserver_master_addrs=127.24.31.254:44321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:55.459519 10772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:55.460847 10772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:55.475917 10802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:55.475953 10803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:55.476861 10806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:55.477358 10772 server_base.cc:1034] running on GCE node
I20260809 23:02:55.478096 10772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:55.479132 10772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:55.480368 10772 hybrid_clock.cc:648] HybridClock initialized: now 1786316575480296 us; error 72 us; skew 500 ppm
I20260809 23:02:55.480702 10772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:55 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:55.483995 10772 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:02:55.485721 10772 webserver.cc:460] Webserver started at http://127.24.31.193:34369/ using document root <none> and password file <none>
I20260809 23:02:55.486438 10772 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:55.486598 10772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:55.492321 10772 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 23:02:55.495139 10815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:55.496582 10772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 23:02:55.496764 10772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "e43fc3fedbf14297bc8a264650d5344a"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "ad2491ce1b50b415648e9138ef5a1c48"
server_key_iv: "062a53251c3f60a36bea0ebe9f9287a3"
server_key_version: "encryptionkey@0"
I20260809 23:02:55.497960 10772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:55.523501 10772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:55.528489 10772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:55.528798 10772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:55.530061 10772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:55.531486 10772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:55.531648 10772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:55.531785 10772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:55.531889 10772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:55.569526 10772 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:37095
I20260809 23:02:55.569612 10950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:37095 every 8 connection(s)
I20260809 23:02:55.571002 10772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:02:55.577872 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 10772
I20260809 23:02:55.578750 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:38193
--local_ip_for_outbound_sockets=127.24.31.194
--tserver_master_addrs=127.24.31.254:44321
--webserver_port=36979
--webserver_interface=127.24.31.194
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:02:55 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:55.598456 10955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:55.574232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:40665)
0809 23:02:55.575142 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:55.575186 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:55.576976 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:55.578398 (+ 1422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:55.578409 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:55.579199 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:55.580010 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:55.580039 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:55.581793 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:55.581801 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:55.583925 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:55.583957 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:55.584996 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:55.585005 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:55.585355 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:55.586156 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:55.586193 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:55.589303 (+ 3110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:55.594224 (+ 4921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:55.594232 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:55.594244 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:55.594698 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:55.595204 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:55.595208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:55.595210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:55.595406 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:55.596370 (+ 964us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:55.596375 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:55.596662 (+ 287us) client_negotiation.cc:769] Sending connection context
0809 23:02:55.596892 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 23:02:55.597425 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":390,"threads_started":1}
I20260809 23:02:55.598896 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:55.573988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:40665 (local address 127.24.31.254:44321)
0809 23:02:55.574609 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:55.574620 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:55.575324 (+ 704us) server_negotiation.cc:407] Connection header received
0809 23:02:55.577497 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:55.577509 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:55.577817 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:55.578180 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:55.580254 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:55.581612 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:55.584111 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:55.584745 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:55.584905 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:55.589613 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:55.589625 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:55.589648 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:55.589708 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:55.594045 (+ 4337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:55.594856 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:55.594868 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:55.594879 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:55.595085 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:55.595519 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:55.595526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:55.595532 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:55.596224 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:55.596443 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:55.597196 (+ 753us) server_negotiation.cc:299] Negotiation successful
0809 23:02:55.597648 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":299,"threads_started":1}
I20260809 23:02:55.601639 10951 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44321
I20260809 23:02:55.601967 10951 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:55.602890 10951 heartbeater.cc:511] Master 127.24.31.254:44321 requested a full tablet report, sending...
I20260809 23:02:55.605409 10726 ts_manager.cc:194] Registered new tserver with Master: e43fc3fedbf14297bc8a264650d5344a (127.24.31.193:37095)
I20260809 23:02:55.606896 10726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:40665
W20260809 23:02:55.913573 10957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:55.913844 10957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:55.913995 10957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:55.921077 10957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:55.921211 10957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:55.921401 10957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:02:55.931121 10957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:38193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=36979
--tserver_master_addrs=127.24.31.254:44321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:55.932473 10957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:55.933808 10957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:55.946703 10964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:55.946750 10966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:55.946759 10963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:55.947546 10957 server_base.cc:1034] running on GCE node
I20260809 23:02:55.948067 10957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:55.948980 10957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:55.950201 10957 hybrid_clock.cc:648] HybridClock initialized: now 1786316575950162 us; error 54 us; skew 500 ppm
I20260809 23:02:55.950528 10957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:55 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:55.953576 10957 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:02:55.954952 10957 webserver.cc:460] Webserver started at http://127.24.31.194:36979/ using document root <none> and password file <none>
I20260809 23:02:55.955606 10957 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:55.955771 10957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:55.961009 10957 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:02:55.963711 10974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:55.965052 10957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:02:55.965251 10957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "90ec54e6dc8a4ead821d9eb5acd8950f"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "f56f100742d7a77e6d84ea0ff7a7dbd3"
server_key_iv: "7925ace75247ce55263729cd31dd45cf"
server_key_version: "encryptionkey@0"
I20260809 23:02:55.966300 10957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:55.978853 10957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:55.982862 10957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:55.983158 10957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:55.984120 10957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:55.985531 10957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:55.985656 10957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:55.985800 10957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:55.985893 10957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:56.021309 10957 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:38193
I20260809 23:02:56.021353 11102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:38193 every 8 connection(s)
I20260809 23:02:56.023049 10957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:02:56.032095 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 10957
I20260809 23:02:56.032853 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:34761
--local_ip_for_outbound_sockets=127.24.31.195
--tserver_master_addrs=127.24.31.254:44321
--webserver_port=46669
--webserver_interface=127.24.31.195
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:02:56 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316575, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:56.044418 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.025710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:34883 (local address 127.24.31.254:44321)
0809 23:02:56.025941 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.025949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.027251 (+ 1302us) server_negotiation.cc:407] Connection header received
0809 23:02:56.028992 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.028997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.029064 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.029223 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:56.030617 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.031295 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.033143 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.033714 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.033795 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.037350 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:56.037359 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:56.037367 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:56.037415 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:56.040992 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.041879 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.041887 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.041893 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.042056 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.042591 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.042598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.042600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.043011 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:56.043190 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.044009 (+ 819us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.044187 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 23:02:56.045874 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.026305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:34883)
0809 23:02:56.027079 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:56.027115 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:56.028717 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:56.029429 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:56.029437 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:56.029921 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:56.030453 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.030470 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.031430 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.031433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.032928 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.032939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.034057 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.034061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.034267 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.034944 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:56.034965 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:56.037092 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:56.041137 (+ 4045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:56.041147 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:56.041186 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:56.041741 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:56.042182 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:56.042189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:56.042194 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:56.042437 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:56.043300 (+ 863us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:56.043306 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:56.043587 (+ 281us) client_negotiation.cc:769] Sending connection context
0809 23:02:56.043899 (+ 312us) client_negotiation.cc:240] Negotiation successful
0809 23:02:56.044438 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":344,"threads_started":1}
I20260809 23:02:56.047405 11103 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44321
I20260809 23:02:56.047712 11103 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:56.048458 11103 heartbeater.cc:511] Master 127.24.31.254:44321 requested a full tablet report, sending...
I20260809 23:02:56.049906 10726 ts_manager.cc:194] Registered new tserver with Master: 90ec54e6dc8a4ead821d9eb5acd8950f (127.24.31.194:38193)
I20260809 23:02:56.050760 10726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:34883
W20260809 23:02:56.372248 11109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:02:56.372534 11109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:02:56.372728 11109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:02:56.380139 11109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:02:56.380300 11109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:02:56.380494 11109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:02:56.390280 11109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:34761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=46669
--tserver_master_addrs=127.24.31.254:44321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:02:56.391575 11109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:02:56.392863 11109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:02:56.407372 11116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:56.407402 11119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:02:56.407368 11117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:02:56.408995 11109 server_base.cc:1034] running on GCE node
I20260809 23:02:56.409677 11109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:02:56.410674 11109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:02:56.411885 11109 hybrid_clock.cc:648] HybridClock initialized: now 1786316576411851 us; error 102 us; skew 500 ppm
I20260809 23:02:56.412202 11109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:02:56 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316576, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:02:56.415391 11109 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:02:56.416841 11109 webserver.cc:460] Webserver started at http://127.24.31.195:46669/ using document root <none> and password file <none>
I20260809 23:02:56.417575 11109 fs_manager.cc:359] Metadata directory not provided
I20260809 23:02:56.417734 11109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:02:56.423166 11109 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 23:02:56.425907 11126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:56.427152 11109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:02:56.427327 11109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "a3f80c45f7bb41bb808addac45ad4592"
format_stamp: "Formatted at 2026-08-09 23:02:33 on dist-test-slave-vxj2"
server_key: "8b3292723c1de0c34a8d3e78c0311b14"
server_key_iv: "4e487772d50c513e748f6410feb3fd48"
server_key_version: "encryptionkey@0"
I20260809 23:02:56.428272 11109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:02:56.452250 11109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:02:56.456053 11109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:02:56.456326 11109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:02:56.457376 11109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:02:56.458727 11109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:02:56.458819 11109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:56.458892 11109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:02:56.458956 11109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:02:56.492705 11109 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:34761
I20260809 23:02:56.492782 11252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:34761 every 8 connection(s)
I20260809 23:02:56.494233 11109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:02:56.504472 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 11109
Aug 09 23:02:56 dist-test-slave-vxj2 krb5kdc[10041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316576, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:02:56.516042 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.496873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:52585 (local address 127.24.31.254:44321)
0809 23:02:56.497229 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.497234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.498309 (+ 1075us) server_negotiation.cc:407] Connection header received
0809 23:02:56.500139 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.500146 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.500241 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.500355 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:56.501881 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.502632 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.504154 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.504574 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.504712 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.509687 (+ 4975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:56.509691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:56.509695 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:56.509726 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:56.512598 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.513566 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.513571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.513573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.513703 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.514143 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.514147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.514149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.514529 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:56.514672 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.515588 (+ 916us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.515751 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260809 23:02:56.517620 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.497440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:52585)
0809 23:02:56.498133 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:56.498170 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:56.499877 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:56.500596 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:56.500605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:56.501101 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:02:56.501712 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.501729 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.502803 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.502810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.504008 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.504020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.504901 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.504910 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.505270 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.506034 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:02:56.506073 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:02:56.509447 (+ 3374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:02:56.512732 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:56.512741 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:56.512765 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:56.513421 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:56.513823 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:02:56.513828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:02:56.513832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:02:56.514014 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:02:56.514679 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:02:56.514687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:02:56.515170 (+ 483us) client_negotiation.cc:769] Sending connection context
0809 23:02:56.515500 (+ 330us) client_negotiation.cc:240] Negotiation successful
0809 23:02:56.516137 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":277,"threads_started":1}
I20260809 23:02:56.519256 11253 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44321
I20260809 23:02:56.519665 11253 heartbeater.cc:464] Registering TS with master...
I20260809 23:02:56.520411 11253 heartbeater.cc:511] Master 127.24.31.254:44321 requested a full tablet report, sending...
I20260809 23:02:56.522166 10726 ts_manager.cc:194] Registered new tserver with Master: a3f80c45f7bb41bb808addac45ad4592 (127.24.31.195:34761)
I20260809 23:02:56.523144 10726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:52585
I20260809 23:02:56.524019 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.505600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33446 (local address 127.24.31.254:44321)
0809 23:02:56.508289 (+ 2689us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.508293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.508308 (+ 15us) server_negotiation.cc:407] Connection header received
0809 23:02:56.508345 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.508348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.508423 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.508535 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:56.510668 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.511704 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.513276 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.513714 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.513848 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.517527 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:56.517531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:56.517535 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:56.517563 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:56.520698 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.521614 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.521618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.521620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.521730 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.522192 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.522196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.522198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.522588 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:56.522863 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.523238 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.523469 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2601,"thread_start_us":2482,"threads_started":1}
I20260809 23:02:56.526827 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:02:56.533718 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.527805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.24.31.254:44321)
0809 23:02:56.528014 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.528018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.528406 (+ 388us) server_negotiation.cc:407] Connection header received
0809 23:02:56.528575 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.528579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.528699 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.528817 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:02:56.529327 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.530050 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.531532 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.532126 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.532265 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.532738 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:02:56.533090 (+ 352us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 23:02:56.533497 (+ 407us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33450: token expired
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 23:02:56.533812 11258 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33450: token expired
W20260809 23:02:56.534499 10634 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.31.254:44321)
I20260809 23:02:56.547915 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.535003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.24.31.254:44321)
0809 23:02:56.535165 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.535172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.535210 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:02:56.535413 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.535419 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.535483 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.535581 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:56.537059 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.537871 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.539261 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.539670 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.539799 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.542613 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:56.542618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:56.542623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:56.542664 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:56.545100 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.545892 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.545899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.545903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.546046 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.546431 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.546437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.546440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.546751 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:56.547016 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.547354 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.547573 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 23:02:56.551369 10726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33462:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:02:56.558687 10726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:02:56.586833 11268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.570727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:38193 (local address 127.0.0.1:56204)
0809 23:02:56.571565 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:56.571607 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:56.571763 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:56.579004 (+ 7241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:56.579015 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:56.579058 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:56.579858 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.579876 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.582170 (+ 2294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.582176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.584043 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.584056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.585745 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.585751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.585978 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.586038 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 23:02:56.586186 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 23:02:56.586382 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":4,"thread_start_us":9804,"threads_started":1}
I20260809 23:02:56.587720 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.571911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56204 (local address 127.24.31.194:38193)
0809 23:02:56.578196 (+ 6285us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.578212 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.578250 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:02:56.578390 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.578402 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.578708 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.580116 (+ 1408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:56.580430 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.581977 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.584206 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.585140 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.585354 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.585574 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.587184 (+ 1610us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.587385 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":281,"threads_started":1}
I20260809 23:02:56.587833 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.570802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:34761 (local address 127.0.0.1:43084)
0809 23:02:56.578820 (+ 8018us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:56.578843 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:56.578968 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:56.579874 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:56.579878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:56.579896 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:56.580187 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.580193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.582880 (+ 2687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.582885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.585134 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.585166 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.587022 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.587039 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.587235 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.587272 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 23:02:56.587356 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 23:02:56.587477 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7884,"thread_start_us":372,"threads_started":1}
I20260809 23:02:56.588459 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.571523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43084 (local address 127.24.31.195:34761)
0809 23:02:56.572709 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.572720 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.579099 (+ 6379us) server_negotiation.cc:407] Connection header received
0809 23:02:56.579200 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.579207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.579441 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.579635 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:56.580427 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.582490 (+ 2063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.585311 (+ 2821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.586768 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.586948 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.588015 (+ 1067us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.588090 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.588216 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":998,"thread_start_us":1240,"threads_started":1}
I20260809 23:02:56.589334 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.573091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.24.31.193:37095)
0809 23:02:56.578340 (+ 5249us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.578351 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.580942 (+ 2591us) server_negotiation.cc:407] Connection header received
0809 23:02:56.581075 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.581082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.581302 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.581496 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:56.582058 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.583560 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.586454 (+ 2894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.587807 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.587944 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.588172 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.588746 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.588931 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5026,"thread_start_us":4843,"threads_started":1}
I20260809 23:02:56.589748 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.572686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37095 (local address 127.0.0.1:43484)
0809 23:02:56.580803 (+ 8117us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:56.580822 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:56.580950 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:56.581499 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:56.581504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:56.581529 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:56.581880 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.581894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.583740 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.583745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.586276 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.586289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.588377 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.588382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.588538 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.588579 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 23:02:56.589266 (+ 687us) client_negotiation.cc:240] Negotiation successful
0809 23:02:56.589402 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7995,"mutex_wait_us":319,"thread_start_us":5873,"threads_started":1}
I20260809 23:02:56.592072 11030 tablet_service.cc:1467] Processing CreateTablet for tablet 6632e33e324b43ae9deed4fdfbc90bac (DEFAULT_TABLE table=test-table [id=2043dab1f6b44d1dbb77daa4f0d58fc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:56.592510 11183 tablet_service.cc:1467] Processing CreateTablet for tablet 6632e33e324b43ae9deed4fdfbc90bac (DEFAULT_TABLE table=test-table [id=2043dab1f6b44d1dbb77daa4f0d58fc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:56.593375 11030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6632e33e324b43ae9deed4fdfbc90bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:56.593890 11183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6632e33e324b43ae9deed4fdfbc90bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:56.593459 10874 tablet_service.cc:1467] Processing CreateTablet for tablet 6632e33e324b43ae9deed4fdfbc90bac (DEFAULT_TABLE table=test-table [id=2043dab1f6b44d1dbb77daa4f0d58fc4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:02:56.594820 10874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6632e33e324b43ae9deed4fdfbc90bac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:02:56.605995 11273 tablet_bootstrap.cc:492] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: Bootstrap starting.
I20260809 23:02:56.609391 11273 tablet_bootstrap.cc:654] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:56.610708 11275 tablet_bootstrap.cc:492] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: Bootstrap starting.
I20260809 23:02:56.611047 11273 log.cc:824] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:56.611102 10951 heartbeater.cc:503] Master 127.24.31.254:44321 was elected leader, sending a full tablet report...
I20260809 23:02:56.616119 11277 tablet_bootstrap.cc:492] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: Bootstrap starting.
I20260809 23:02:56.617228 11275 tablet_bootstrap.cc:654] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:56.619283 11275 log.cc:824] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:56.620327 11277 tablet_bootstrap.cc:654] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: Neither blocks nor log segments found. Creating new log.
I20260809 23:02:56.621644 11277 log.cc:824] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: Log is configured to *not* fsync() on all Append() calls
I20260809 23:02:56.622968 11275 tablet_bootstrap.cc:492] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: No bootstrap required, opened a new log
I20260809 23:02:56.622964 11273 tablet_bootstrap.cc:492] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: No bootstrap required, opened a new log
I20260809 23:02:56.623322 11275 ts_tablet_manager.cc:1397] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260809 23:02:56.623387 11273 ts_tablet_manager.cc:1397] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.016s
I20260809 23:02:56.624713 11277 tablet_bootstrap.cc:492] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: No bootstrap required, opened a new log
I20260809 23:02:56.624987 11277 ts_tablet_manager.cc:1397] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 23:02:56.631033 11275 raft_consensus.cc:348] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } }
I20260809 23:02:56.631448 11275 raft_consensus.cc:374] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:56.631558 11275 raft_consensus.cc:729] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e43fc3fedbf14297bc8a264650d5344a, State: Initialized, Role: FOLLOWER
I20260809 23:02:56.632274 11275 consensus_queue.cc:260] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } }
I20260809 23:02:56.634523 11275 ts_tablet_manager.cc:1428] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260809 23:02:56.634446 11273 raft_consensus.cc:348] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } }
I20260809 23:02:56.634928 11273 raft_consensus.cc:374] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:56.635030 11273 raft_consensus.cc:729] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ec54e6dc8a4ead821d9eb5acd8950f, State: Initialized, Role: FOLLOWER
I20260809 23:02:56.635639 11273 consensus_queue.cc:260] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } }
I20260809 23:02:56.635663 11277 raft_consensus.cc:348] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } }
I20260809 23:02:56.636075 11277 raft_consensus.cc:374] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:02:56.636262 11277 raft_consensus.cc:729] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f80c45f7bb41bb808addac45ad4592, State: Initialized, Role: FOLLOWER
I20260809 23:02:56.636797 11103 heartbeater.cc:503] Master 127.24.31.254:44321 was elected leader, sending a full tablet report...
I20260809 23:02:56.637014 11277 consensus_queue.cc:260] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } }
I20260809 23:02:56.637455 11273 ts_tablet_manager.cc:1428] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260809 23:02:56.642024 11253 heartbeater.cc:503] Master 127.24.31.254:44321 was elected leader, sending a full tablet report...
I20260809 23:02:56.642395 11277 ts_tablet_manager.cc:1428] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
I20260809 23:02:56.655422 11283 raft_consensus.cc:482] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:02:56.655659 11283 raft_consensus.cc:504] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } }
I20260809 23:02:56.657276 11283 leader_election.cc:288] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3f80c45f7bb41bb808addac45ad4592 (127.24.31.195:34761), e43fc3fedbf14297bc8a264650d5344a (127.24.31.193:37095)
I20260809 23:02:56.664143 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.657910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:48585 (local address 127.24.31.193:37095)
0809 23:02:56.658147 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.658153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.658680 (+ 527us) server_negotiation.cc:407] Connection header received
0809 23:02:56.659015 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.659023 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.659096 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.659208 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:56.659716 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.660469 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.662433 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.663209 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.663401 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.663480 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.663754 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.663891 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 23:02:56.664310 11285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.657673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37095 (local address 127.24.31.194:48585)
0809 23:02:56.658557 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:56.658578 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:56.658830 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:56.659239 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:56.659244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:56.659281 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:56.659574 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.659582 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.660613 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.660618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.662278 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.662291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.663431 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.663437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.663587 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.663610 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 23:02:56.663720 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 23:02:56.663847 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":59,"thread_start_us":670,"threads_started":1}
I20260809 23:02:56.664359 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.657739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:42683 (local address 127.24.31.195:34761)
0809 23:02:56.657959 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.657966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.659183 (+ 1217us) server_negotiation.cc:407] Connection header received
0809 23:02:56.659377 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.659380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.659438 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.659535 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:02:56.660159 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.660858 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.662635 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.663416 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.663603 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.663678 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.663972 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.664101 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 23:02:56.664620 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.658211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:34761 (local address 127.24.31.194:42683)
0809 23:02:56.659083 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 23:02:56.659097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:02:56.659230 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:02:56.659564 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:02:56.659568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:02:56.659592 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:02:56.659996 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.660011 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.660991 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.660997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.662480 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:02:56.662492 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.663645 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.663651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:02:56.663810 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.663841 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 23:02:56.663948 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 23:02:56.664076 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":497,"threads_started":1}
I20260809 23:02:56.665665 11204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6632e33e324b43ae9deed4fdfbc90bac" candidate_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f80c45f7bb41bb808addac45ad4592" is_pre_election: true
I20260809 23:02:56.665661 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6632e33e324b43ae9deed4fdfbc90bac" candidate_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e43fc3fedbf14297bc8a264650d5344a" is_pre_election: true
I20260809 23:02:56.666150 11204 raft_consensus.cc:2454] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90ec54e6dc8a4ead821d9eb5acd8950f in term 0.
I20260809 23:02:56.666150 10898 raft_consensus.cc:2454] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90ec54e6dc8a4ead821d9eb5acd8950f in term 0.
I20260809 23:02:56.666877 10977 leader_election.cc:302] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 90ec54e6dc8a4ead821d9eb5acd8950f, a3f80c45f7bb41bb808addac45ad4592, e43fc3fedbf14297bc8a264650d5344a; no voters:
I20260809 23:02:56.667342 11283 raft_consensus.cc:2781] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:02:56.667493 11283 raft_consensus.cc:482] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:02:56.667621 11283 raft_consensus.cc:3037] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:56.670042 11283 raft_consensus.cc:504] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } }
I20260809 23:02:56.670753 11283 leader_election.cc:288] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [CANDIDATE]: Term 1 election: Requested vote from peers a3f80c45f7bb41bb808addac45ad4592 (127.24.31.195:34761), e43fc3fedbf14297bc8a264650d5344a (127.24.31.193:37095)
I20260809 23:02:56.671221 11204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6632e33e324b43ae9deed4fdfbc90bac" candidate_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3f80c45f7bb41bb808addac45ad4592"
I20260809 23:02:56.671221 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6632e33e324b43ae9deed4fdfbc90bac" candidate_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e43fc3fedbf14297bc8a264650d5344a"
I20260809 23:02:56.671429 10898 raft_consensus.cc:3037] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:56.671420 11204 raft_consensus.cc:3037] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:02:56.673743 11204 raft_consensus.cc:2454] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90ec54e6dc8a4ead821d9eb5acd8950f in term 1.
I20260809 23:02:56.673769 10898 raft_consensus.cc:2454] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90ec54e6dc8a4ead821d9eb5acd8950f in term 1.
I20260809 23:02:56.674192 10976 leader_election.cc:302] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ec54e6dc8a4ead821d9eb5acd8950f, a3f80c45f7bb41bb808addac45ad4592; no voters:
I20260809 23:02:56.674497 11283 raft_consensus.cc:2781] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:02:56.674943 11283 raft_consensus.cc:686] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 1 LEADER]: Becoming Leader. State: Replica: 90ec54e6dc8a4ead821d9eb5acd8950f, State: Running, Role: LEADER
I20260809 23:02:56.675407 11283 consensus_queue.cc:237] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } }
I20260809 23:02:56.680119 10726 catalog_manager.cc:5495] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f reported cstate change: term changed from 0 to 1, leader changed from <none> to 90ec54e6dc8a4ead821d9eb5acd8950f (127.24.31.194). New cstate: current_term: 1 leader_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ec54e6dc8a4ead821d9eb5acd8950f" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 38193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 } health_report { overall_health: UNKNOWN } } }
I20260809 23:02:56.714668 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.709394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56214 (local address 127.24.31.194:38193)
0809 23:02:56.709608 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.709613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.709736 (+ 123us) server_negotiation.cc:407] Connection header received
0809 23:02:56.709974 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.709979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.710072 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.710205 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:02:56.710526 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.711370 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.712720 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.713105 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.713276 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.713713 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:02:56.714210 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:02:56.714318 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.714394 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.714460 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 23:02:56.723721 11204 raft_consensus.cc:1261] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [term 1 FOLLOWER]: Refusing update from remote peer 90ec54e6dc8a4ead821d9eb5acd8950f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:02:56.723874 10898 raft_consensus.cc:1261] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [term 1 FOLLOWER]: Refusing update from remote peer 90ec54e6dc8a4ead821d9eb5acd8950f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:02:56.724797 11283 consensus_queue.cc:1035] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3f80c45f7bb41bb808addac45ad4592" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 34761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:02:56.725100 11287 consensus_queue.cc:1035] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e43fc3fedbf14297bc8a264650d5344a" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:02:56.745828 11254 tablet.cc:2369] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:02:56.746803 11291 mvcc.cc:205] Tried to move back new op lower bound from 7316752698248654848 to 7316752698066063360. Current Snapshot: MvccSnapshot[applied={T|T < 7316752698248654848}]
I20260809 23:02:56.775893 10726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:02:56.776300 10726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33462:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:02:56.781208 10726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Sending DeleteTablet for 3 replicas of tablet 6632e33e324b43ae9deed4fdfbc90bac
I20260809 23:02:56.782677 11183 tablet_service.cc:1514] Processing DeleteTablet for tablet 6632e33e324b43ae9deed4fdfbc90bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:56 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:43084
I20260809 23:02:56.783356 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 10772
I20260809 23:02:56.783514 10874 tablet_service.cc:1514] Processing DeleteTablet for tablet 6632e33e324b43ae9deed4fdfbc90bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:56 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:43484
I20260809 23:02:56.790130 11304 tablet_replica.cc:330] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: stopping tablet replica
I20260809 23:02:56.790541 11030 tablet_service.cc:1514] Processing DeleteTablet for tablet 6632e33e324b43ae9deed4fdfbc90bac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:02:56 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:56204
I20260809 23:02:56.791288 11304 raft_consensus.cc:2229] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:02:56.792457 11304 raft_consensus.cc:2258] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:56.796854 11305 tablet_replica.cc:330] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: stopping tablet replica
I20260809 23:02:56.797544 11305 raft_consensus.cc:2229] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:02:56.798218 11305 raft_consensus.cc:2258] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:56.798854 11306 tablet_replica.cc:330] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: stopping tablet replica
I20260809 23:02:56.799626 11306 raft_consensus.cc:2229] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:02:56.802019 11306 raft_consensus.cc:2258] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:02:56.802860 11305 ts_tablet_manager.cc:1905] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:56.805209 11306 ts_tablet_manager.cc:1905] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:56.806833 11304 ts_tablet_manager.cc:1905] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:02:56.804957 11272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.784209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.24.31.193:37095)
0809 23:02:56.784427 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.784435 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.785053 (+ 618us) server_negotiation.cc:407] Connection header received
0809 23:02:56.785121 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.785126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.785238 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.785346 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:56.786969 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.788597 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.791281 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.791992 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.792124 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.796006 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:56.796027 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:56.796046 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:56.796091 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:56.800549 (+ 4458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.801529 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.801539 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.801543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.801698 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.802218 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.802223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.802227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.802736 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:56.804182 (+ 1446us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.804358 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.804590 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 23:02:56.810654 10946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:56.814420 11304 ts_tablet_manager.cc:1918] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:56.814651 11304 log.cc:1196] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/6632e33e324b43ae9deed4fdfbc90bac
I20260809 23:02:56.815351 11304 ts_tablet_manager.cc:1939] T 6632e33e324b43ae9deed4fdfbc90bac P a3f80c45f7bb41bb808addac45ad4592: Deleting consensus metadata
I20260809 23:02:56.817468 10707 catalog_manager.cc:4841] TS a3f80c45f7bb41bb808addac45ad4592 (127.24.31.195:34761): tablet 6632e33e324b43ae9deed4fdfbc90bac (table test-table [id=2043dab1f6b44d1dbb77daa4f0d58fc4]) successfully deleted
I20260809 23:02:56.820851 11306 ts_tablet_manager.cc:1918] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:56.821089 11306 log.cc:1196] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/6632e33e324b43ae9deed4fdfbc90bac
I20260809 23:02:56.822029 11306 ts_tablet_manager.cc:1939] T 6632e33e324b43ae9deed4fdfbc90bac P 90ec54e6dc8a4ead821d9eb5acd8950f: Deleting consensus metadata
I20260809 23:02:56.824204 10706 catalog_manager.cc:4841] TS 90ec54e6dc8a4ead821d9eb5acd8950f (127.24.31.194:38193): tablet 6632e33e324b43ae9deed4fdfbc90bac (table test-table [id=2043dab1f6b44d1dbb77daa4f0d58fc4]) successfully deleted
I20260809 23:02:56.916064 11305 ts_tablet_manager.cc:1918] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:02:56.916285 11305 log.cc:1196] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/6632e33e324b43ae9deed4fdfbc90bac
I20260809 23:02:56.916750 11305 ts_tablet_manager.cc:1939] T 6632e33e324b43ae9deed4fdfbc90bac P e43fc3fedbf14297bc8a264650d5344a: Deleting consensus metadata
I20260809 23:02:56.918429 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 10772
I20260809 23:02:56.918500 10708 catalog_manager.cc:4841] TS e43fc3fedbf14297bc8a264650d5344a (127.24.31.193:37095): tablet 6632e33e324b43ae9deed4fdfbc90bac (table test-table [id=2043dab1f6b44d1dbb77daa4f0d58fc4]) successfully deleted
I20260809 23:02:56.940989 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 10957
I20260809 23:02:56.960546 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:56.942071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56226 (local address 127.24.31.194:38193)
0809 23:02:56.942258 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:56.942263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:56.942280 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:02:56.942347 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:56.942350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:56.942418 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:56.942535 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:56.944078 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.945024 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.946449 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:56.946982 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:56.947139 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:56.953786 (+ 6647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:56.953809 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:56.953825 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:56.953875 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:56.957066 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.958048 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.958063 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.958067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.958211 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:56.958664 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:56.958668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:56.958670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:56.959085 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:56.959456 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:56.959763 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 23:02:56.960067 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 23:02:56.961378 11098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:57.068879 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 10957
I20260809 23:02:57.090210 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 11109
I20260809 23:02:57.104247 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:02:57.091034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43090 (local address 127.24.31.195:34761)
0809 23:02:57.091222 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 23:02:57.091230 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:02:57.091255 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:02:57.091540 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:02:57.091546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:02:57.091632 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:02:57.091744 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:02:57.093098 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:57.093975 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:57.095306 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:02:57.095725 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:02:57.095862 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:02:57.098599 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:02:57.098612 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:02:57.098628 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:02:57.098658 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:02:57.101434 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:57.102121 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:57.102137 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:57.102141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:57.102277 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:02:57.102665 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:02:57.102671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:02:57.102675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:02:57.103031 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:02:57.103339 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 23:02:57.103671 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 23:02:57.103895 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 23:02:57.104915 11247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:57.196106 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 11109
I20260809 23:02:57.216409 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 10683
I20260809 23:02:57.217763 10762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:02:57.334708 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 10683
I20260809 23:03:17.360299 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44321
--webserver_interface=127.24.31.254
--webserver_port=43085
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:44321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:17.720073 11311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:17.720366 11311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:17.720466 11311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:17.728014 11311 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 23:03:17.728121 11311 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 23:03:17.728166 11311 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 23:03:17.728227 11311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:03:17.728283 11311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:17.728360 11311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:17.728418 11311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:03:17.728473 11311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:03:17.738327 11311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:44321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=43085
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:17.739559 11311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:17.740922 11311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:17.753319 11319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:17.753437 11318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:17.753437 11322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:17.754856 11311 server_base.cc:1034] running on GCE node
I20260809 23:03:17.755622 11311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:17.757018 11311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:17.758282 11311 hybrid_clock.cc:648] HybridClock initialized: now 1786316597758204 us; error 47 us; skew 500 ppm
I20260809 23:03:17.758630 11311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:17 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316597, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:17.762020 11311 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:03:17.763526 11311 webserver.cc:460] Webserver started at http://127.24.31.254:43085/ using document root <none> and password file <none>
I20260809 23:03:17.764236 11311 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:17.764374 11311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:17.770191 11311 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 23:03:17.773449 11329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:17.774751 11311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 23:03:17.774961 11311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "8b63c01156b04ad0b7caeef1544dabbf"
format_stamp: "Formatted at 2026-08-09 23:02:31 on dist-test-slave-vxj2"
server_key: "870d2a9699c061b0bbdd71ca3dd29a98"
server_key_iv: "4629e8483d6ca957ea169eb3a49c4484"
server_key_version: "encryptionkey@0"
I20260809 23:03:17.775995 11311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:17.801865 11311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:17.806739 11311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:17.807066 11311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:17.827988 11311 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:44321
I20260809 23:03:17.828027 11391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:44321 every 8 connection(s)
I20260809 23:03:17.829551 11311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:03:17.836356 11392 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:03:17.839807 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 11311
I20260809 23:03:17.839880 11392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Bootstrap starting.
I20260809 23:03:17.840903 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:37095
--local_ip_for_outbound_sockets=127.24.31.193
--tserver_master_addrs=127.24.31.254:44321
--webserver_port=34369
--webserver_interface=127.24.31.193
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:03:17.864840 11392 log.cc:824] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:17.880978 11392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:03:17.881556 11392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Bootstrap complete.
I20260809 23:03:17.890854 11392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:03:17.891760 11392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b63c01156b04ad0b7caeef1544dabbf, State: Initialized, Role: FOLLOWER
I20260809 23:03:17.892401 11392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:03:17.892638 11392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:03:17.892755 11392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:03:17.892961 11392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 2 FOLLOWER]: Advancing to term 3
I20260809 23:03:17.896842 11392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:03:17.897378 11392 leader_election.cc:302] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b63c01156b04ad0b7caeef1544dabbf; no voters:
I20260809 23:03:17.898097 11392 leader_election.cc:288] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 23:03:17.898229 11398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 3 FOLLOWER]: Leader election won for term 3
I20260809 23:03:17.900617 11392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:03:17.905390 11398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [term 3 LEADER]: Becoming Leader. State: Replica: 8b63c01156b04ad0b7caeef1544dabbf, State: Running, Role: LEADER
I20260809 23:03:17.906018 11398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } }
I20260809 23:03:17.908892 11399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8b63c01156b04ad0b7caeef1544dabbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } } }
I20260809 23:03:17.909338 11399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:17.909061 11400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b63c01156b04ad0b7caeef1544dabbf. Latest consensus state: current_term: 3 leader_uuid: "8b63c01156b04ad0b7caeef1544dabbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b63c01156b04ad0b7caeef1544dabbf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44321 } } }
I20260809 23:03:17.909502 11400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:17.914500 11410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:03:17.929885 11410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:03:17.931841 11410 catalog_manager.cc:1259] Loaded cluster ID: d183f5e24a754cde910165a9df860b4c
I20260809 23:03:17.932317 11410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:03:17.937650 11410 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:03:17.939843 11410 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Loaded TSK: 0
I20260809 23:03:17.940109 11410 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8b63c01156b04ad0b7caeef1544dabbf: Loaded TSK: 1
F20260809 23:03:17.941607 11410 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f93b188e9ad google::LogMessage::Fail() at ??:0
@ 0x7f93b189188a google::LogMessage::SendToLog() at ??:0
@ 0x7f93b188e4de google::LogMessage::Flush() at ??:0
@ 0x7f93b188fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f93b55f27d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f93b55e173e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f93c72bd1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f93c72bc783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f93c73482a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f93c73480c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f93c73925e9 std::function<>::operator()() at ??:0
@ 0x7f93c72c0cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f93c72bec36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f93c7346610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f93c7346439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f93b44da345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f93b44dd91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f93b44dd799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f93b44b8eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f93b77926db start_thread at ??:0
@ 0x7f93b072861f clone at ??:0
W20260809 23:03:18.287024 11395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:18.287357 11395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:18.287535 11395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:18.294988 11395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:18.295143 11395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:18.295316 11395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:03:18.305359 11395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:37095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=34369
--tserver_master_addrs=127.24.31.254:44321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:18.306685 11395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:18.307984 11395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:18.321255 11457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:18.321538 11454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:18.321852 11455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:18.322108 11395 server_base.cc:1034] running on GCE node
I20260809 23:03:18.322886 11395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:18.323778 11395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:18.324977 11395 hybrid_clock.cc:648] HybridClock initialized: now 1786316598324918 us; error 57 us; skew 500 ppm
I20260809 23:03:18.325342 11395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:18 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316598, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:18.328429 11395 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:03:18.329973 11395 webserver.cc:460] Webserver started at http://127.24.31.193:34369/ using document root <none> and password file <none>
I20260809 23:03:18.330578 11395 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:18.330677 11395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:18.335731 11395 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 23:03:18.338315 11464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:18.339593 11395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:03:18.339766 11395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "e43fc3fedbf14297bc8a264650d5344a"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "ad2491ce1b50b415648e9138ef5a1c48"
server_key_iv: "062a53251c3f60a36bea0ebe9f9287a3"
server_key_version: "encryptionkey@0"
I20260809 23:03:18.340725 11395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:18.364369 11395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:18.368863 11395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:18.369187 11395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:18.370260 11395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:18.371671 11395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:18.371770 11395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:18.371932 11395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:18.372025 11395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:18.408362 11395 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:37095
I20260809 23:03:18.408455 11601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:37095 every 8 connection(s)
I20260809 23:03:18.410120 11395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:03:18.413672 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 11395
I20260809 23:03:18.414583 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:38193
--local_ip_for_outbound_sockets=127.24.31.194
--tserver_master_addrs=127.24.31.254:44321
--webserver_port=36979
--webserver_interface=127.24.31.194
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:03:18.416237 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:18.413774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:47341)
0809 23:03:18.414710 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:18.415070 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":331,"threads_started":1}
W20260809 23:03:18.417007 11602 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:44321 (0 consecutive failures): Network error: Failed to ping master at 127.24.31.254:44321: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
I20260809 23:03:18.418001 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:18.417533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:39759)
0809 23:03:18.417743 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:18.417845 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 23:03:18.419060 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:18.418581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:58553)
0809 23:03:18.418764 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:18.418873 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 23:03:18.419327 11602 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:03:18.749493 11605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:18.749769 11605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:18.749895 11605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:18.756913 11605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:18.757021 11605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:18.757216 11605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:03:18.766830 11605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:38193
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=36979
--tserver_master_addrs=127.24.31.254:44321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:18.768110 11605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:18.769415 11605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:18.783066 11611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:18.783035 11614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:18.783102 11612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:18.783960 11605 server_base.cc:1034] running on GCE node
I20260809 23:03:18.784507 11605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:18.785419 11605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:18.786610 11605 hybrid_clock.cc:648] HybridClock initialized: now 1786316598786560 us; error 51 us; skew 500 ppm
I20260809 23:03:18.786866 11605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:18 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316598, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:18.795434 11605 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:03:18.796917 11605 webserver.cc:460] Webserver started at http://127.24.31.194:36979/ using document root <none> and password file <none>
I20260809 23:03:18.797639 11605 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:18.797744 11605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:18.802999 11605 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 23:03:18.805670 11622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:18.807034 11605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 23:03:18.807168 11605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "90ec54e6dc8a4ead821d9eb5acd8950f"
format_stamp: "Formatted at 2026-08-09 23:02:32 on dist-test-slave-vxj2"
server_key: "f56f100742d7a77e6d84ea0ff7a7dbd3"
server_key_iv: "7925ace75247ce55263729cd31dd45cf"
server_key_version: "encryptionkey@0"
I20260809 23:03:18.808094 11605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:18.820545 11605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:18.824685 11605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:18.824913 11605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:18.825867 11605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:18.827126 11605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:18.827212 11605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:18.827289 11605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:18.827354 11605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:18.861881 11605 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:38193
I20260809 23:03:18.861984 11751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:38193 every 8 connection(s)
I20260809 23:03:18.863426 11605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:03:18.868597 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:18.866695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:49011)
0809 23:03:18.867437 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:18.867703 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":287,"threads_started":1}
W20260809 23:03:18.869451 11752 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:44321 (0 consecutive failures): Network error: Failed to ping master at 127.24.31.254:44321: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
I20260809 23:03:18.870290 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:18.869930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:45327)
0809 23:03:18.870089 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:18.870168 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 23:03:18.871243 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:18.870888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:53391)
0809 23:03:18.871037 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:18.871137 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 23:03:18.871490 11752 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:03:18.873517 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 11605
I20260809 23:03:18.874214 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:34761
--local_ip_for_outbound_sockets=127.24.31.195
--tserver_master_addrs=127.24.31.254:44321
--webserver_port=46669
--webserver_interface=127.24.31.195
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:19.219440 11756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:19.219735 11756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:19.219919 11756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:19.227607 11756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:19.227751 11756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:19.227959 11756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:03:19.237774 11756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:34761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=46669
--tserver_master_addrs=127.24.31.254:44321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:19.239042 11756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:19.240453 11756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:19.254385 11761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:19.254429 11764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:19.254457 11762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:19.255442 11756 server_base.cc:1034] running on GCE node
I20260809 23:03:19.256026 11756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:19.256958 11756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:19.258190 11756 hybrid_clock.cc:648] HybridClock initialized: now 1786316599258136 us; error 71 us; skew 500 ppm
I20260809 23:03:19.258550 11756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:19 dist-test-slave-vxj2 krb5kdc[10041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316599, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:19.261916 11756 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:03:19.263386 11756 webserver.cc:460] Webserver started at http://127.24.31.195:46669/ using document root <none> and password file <none>
I20260809 23:03:19.264065 11756 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:19.264225 11756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:19.269688 11756 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260809 23:03:19.272404 11773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:19.273881 11756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:03:19.274070 11756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "a3f80c45f7bb41bb808addac45ad4592"
format_stamp: "Formatted at 2026-08-09 23:02:33 on dist-test-slave-vxj2"
server_key: "8b3292723c1de0c34a8d3e78c0311b14"
server_key_iv: "4e487772d50c513e748f6410feb3fd48"
server_key_version: "encryptionkey@0"
I20260809 23:03:19.275063 11756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:19.299741 11756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:19.304176 11756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:19.304468 11756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:19.305629 11756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:19.306945 11756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:19.307075 11756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:19.307211 11756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:19.307312 11756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:19.342482 11756 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:34761
I20260809 23:03:19.342572 11900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:34761 every 8 connection(s)
I20260809 23:03:19.344046 11756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:03:19.348992 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:19.347134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:50335)
0809 23:03:19.347884 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:19.348145 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":315,"threads_started":1}
W20260809 23:03:19.349761 11901 heartbeater.cc:644] Failed to heartbeat to 127.24.31.254:44321 (0 consecutive failures): Network error: Failed to ping master at 127.24.31.254:44321: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
I20260809 23:03:19.350556 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:19.350234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:38679)
0809 23:03:19.350379 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:19.350445 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 23:03:19.351300 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 11756
I20260809 23:03:19.351460 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:19.351130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:51581)
0809 23:03:19.351252 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:19.351335 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 23:03:19.351697 11901 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:03:19.420807 11905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:19.419931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:47641)
0809 23:03:19.420437 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:19.420537 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":271,"threads_started":1}
I20260809 23:03:19.872983 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:19.872063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:55047)
0809 23:03:19.872537 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:19.872616 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":238,"threads_started":1}
I20260809 23:03:20.353048 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:20.352260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:38511)
0809 23:03:20.352741 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:20.352830 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":262,"threads_started":1}
I20260809 23:03:20.422701 11909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:20.421877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:59895)
0809 23:03:20.422400 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:20.422501 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":217,"threads_started":1}
I20260809 23:03:20.874770 11910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:20.873931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:39267)
0809 23:03:20.874447 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:20.874541 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":257,"threads_started":1}
I20260809 23:03:21.354724 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:21.353969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:55347)
0809 23:03:21.354483 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:21.354563 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":284,"threads_started":1}
I20260809 23:03:21.424350 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:21.423607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:45061)
0809 23:03:21.424083 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:21.424164 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":260,"threads_started":1}
I20260809 23:03:21.876458 11913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:21.875767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:35477)
0809 23:03:21.876219 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:21.876298 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":246,"threads_started":1}
I20260809 23:03:22.356321 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:22.355574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:46495)
0809 23:03:22.356032 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:22.356128 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":264,"threads_started":1}
I20260809 23:03:22.426254 11915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:22.425316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:34845)
0809 23:03:22.425932 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:22.426035 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":341,"threads_started":1}
I20260809 23:03:22.878376 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:22.877520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:34645)
0809 23:03:22.878046 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:22.878128 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":300,"threads_started":1}
I20260809 23:03:23.358122 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:23.357336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:50423)
0809 23:03:23.357850 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:23.357940 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":250,"threads_started":1}
I20260809 23:03:23.428030 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:23.427174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:48105)
0809 23:03:23.427763 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:23.427850 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":305,"threads_started":1}
I20260809 23:03:23.880185 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:23.879383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:60697)
0809 23:03:23.879929 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:23.880010 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":330,"threads_started":1}
I20260809 23:03:24.359911 11921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:24.359071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:51019)
0809 23:03:24.359578 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:24.359688 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":272,"threads_started":1}
I20260809 23:03:24.429860 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:24.429027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:39237)
0809 23:03:24.429580 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:24.429678 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":323,"threads_started":1}
I20260809 23:03:24.881978 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:24.881240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:51909)
0809 23:03:24.881704 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:24.881806 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":231,"threads_started":1}
I20260809 23:03:25.361698 11925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:25.360862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:43233)
0809 23:03:25.361415 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:25.361515 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":251,"threads_started":1}
I20260809 23:03:25.431702 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:25.430877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:44899)
0809 23:03:25.431395 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:25.431492 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":217,"threads_started":1}
I20260809 23:03:25.883746 11927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:25.882957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:46075)
0809 23:03:25.883453 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:25.883532 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":214,"threads_started":1}
I20260809 23:03:26.363487 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:26.362635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:33811)
0809 23:03:26.363169 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:26.363270 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":278,"threads_started":1}
I20260809 23:03:26.433614 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:26.432814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:43887)
0809 23:03:26.433354 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:26.433447 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":236,"threads_started":1}
I20260809 23:03:26.885393 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:26.884619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:38697)
0809 23:03:26.885085 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:26.885226 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":251,"threads_started":1}
I20260809 23:03:27.365376 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:27.364426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:55153)
0809 23:03:27.365009 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:27.365112 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":402,"threads_started":1}
I20260809 23:03:27.435505 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:27.434593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:42771)
0809 23:03:27.435215 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:27.435308 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":301,"threads_started":1}
I20260809 23:03:27.887110 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:27.886324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:43997)
0809 23:03:27.886853 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:27.886934 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":286,"threads_started":1}
I20260809 23:03:28.367009 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:28.366262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:60097)
0809 23:03:28.366756 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:28.366842 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":252,"threads_started":1}
I20260809 23:03:28.437458 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:28.436607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:53213)
0809 23:03:28.437126 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:28.437285 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":302,"threads_started":1}
I20260809 23:03:28.888831 11937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:28.888099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:52381)
0809 23:03:28.888579 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:28.888660 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":271,"threads_started":1}
I20260809 23:03:29.368726 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:29.368011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:36729)
0809 23:03:29.368479 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:29.368563 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":261,"threads_started":1}
I20260809 23:03:29.439323 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:29.438514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:52583)
0809 23:03:29.439009 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:29.439112 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":262,"threads_started":1}
I20260809 23:03:29.890601 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:29.889855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:54225)
0809 23:03:29.890331 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:29.890415 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":231,"threads_started":1}
I20260809 23:03:30.370425 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:30.369697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:40655)
0809 23:03:30.370139 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:30.370235 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":234,"threads_started":1}
I20260809 23:03:30.441427 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:30.440459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:40037)
0809 23:03:30.440969 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:30.441063 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":271,"threads_started":1}
I20260809 23:03:30.892192 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:30.891493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:54471)
0809 23:03:30.891942 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:30.892024 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":225,"threads_started":1}
I20260809 23:03:31.372172 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:31.371388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:35537)
0809 23:03:31.371911 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:31.372000 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":249,"threads_started":1}
I20260809 23:03:31.443380 11949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:31.442508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:36617)
0809 23:03:31.443079 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:31.443176 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":337,"threads_started":1}
I20260809 23:03:31.894107 11951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:31.893260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:47785)
0809 23:03:31.893796 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:31.893926 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":248,"threads_started":1}
I20260809 23:03:32.374029 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:32.373132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:60439)
0809 23:03:32.373722 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:32.373834 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":252,"threads_started":1}
I20260809 23:03:32.445406 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:32.444533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:58029)
0809 23:03:32.445045 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:32.445141 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
I20260809 23:03:32.895884 11954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:32.895056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:41557)
0809 23:03:32.895551 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:32.895655 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":241,"threads_started":1}
I20260809 23:03:33.375701 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:33.374941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:33705)
0809 23:03:33.375432 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:33.375514 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":276,"threads_started":1}
I20260809 23:03:33.447779 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:33.446692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:47325)
0809 23:03:33.447479 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:33.447590 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":421,"threads_started":1}
I20260809 23:03:33.897921 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:33.897102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.194:47345)
0809 23:03:33.897607 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:33.897703 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":295,"threads_started":1}
I20260809 23:03:34.361783 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 11395
I20260809 23:03:34.377767 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:34.376876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.195:42437)
0809 23:03:34.377436 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:34.377558 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":332,"threads_started":1}
I20260809 23:03:34.381415 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:34.363077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46182 (local address 127.24.31.193:37095)
0809 23:03:34.363671 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:34.363694 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:34.363792 (+ 98us) server_negotiation.cc:407] Connection header received
0809 23:03:34.365632 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:34.365642 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:34.366086 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:34.366383 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:34.367966 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:34.369309 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:34.370643 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:34.371333 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:34.371490 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:34.374415 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:34.374422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:34.374443 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:34.374479 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:34.377767 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:34.378528 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:34.378535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:34.378541 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:34.378762 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:34.379152 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:34.379155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:34.379157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:34.379684 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:34.379941 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:34.380475 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 23:03:34.381004 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":896,"threads_started":1}
I20260809 23:03:34.383265 11597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:34.479344 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:34.478228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44321 (local address 127.24.31.193:60489)
0809 23:03:34.478823 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:34.479080 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.254:44321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":324,"threads_started":1}
I20260809 23:03:34.484447 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 11395
I20260809 23:03:34.504699 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 11605
I20260809 23:03:34.523362 11963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:34.505714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47058 (local address 127.24.31.194:38193)
0809 23:03:34.506331 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:34.506355 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:34.506392 (+ 37us) server_negotiation.cc:407] Connection header received
0809 23:03:34.508058 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:34.508067 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:34.508487 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:34.508790 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:34.510323 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:34.511669 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:34.513409 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:34.514080 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:34.514238 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:34.516918 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:34.516926 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:34.516943 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:34.516986 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:34.520144 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:34.520880 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:34.520886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:34.520899 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:34.521074 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:34.521559 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:34.521563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:34.521565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:34.522099 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:34.522354 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:34.522698 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 23:03:34.523068 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":306,"threads_started":1}
I20260809 23:03:34.524778 11747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:34.610561 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 11605
I20260809 23:03:34.633100 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 11756
I20260809 23:03:34.651870 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:34.634331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58874 (local address 127.24.31.195:34761)
0809 23:03:34.634951 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:34.634965 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:34.635007 (+ 42us) server_negotiation.cc:407] Connection header received
0809 23:03:34.636692 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:34.636702 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:34.637117 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:34.637541 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:34.639205 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:34.640414 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:34.641744 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:34.642421 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:34.642618 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:34.645349 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:34.645358 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:34.645372 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:34.645415 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:34.648231 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:34.649002 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:34.649010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:34.649017 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:34.649391 (+ 374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:34.649791 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:34.649799 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:34.649805 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:34.650343 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:34.650648 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:34.651104 (+ 456us) server_negotiation.cc:299] Negotiation successful
0809 23:03:34.651544 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":305,"threads_started":1}
I20260809 23:03:34.653326 11895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:34.737380 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 11756
2026-08-09T23:03:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64821 ms)
[----------] 30 tests from SecurityITest (303325 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:03:34 dist-test-slave-vxj2 krb5kdc[11973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:03:34 dist-test-slave-vxj2 krb5kdc[11973](info): set up 2 sockets
Aug 09 23:03:34 dist-test-slave-vxj2 krb5kdc[11973](info): commencing operation
krb5kdc: starting...
W20260809 23:03:35.940855 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.007s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:03:36 dist-test-slave-vxj2 krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:03:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:03:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:03:36.223239 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:33853
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:33853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:36.561195 11988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:36.561477 11988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:36.561657 11988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:36.569226 11988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:03:36.569355 11988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:36.569442 11988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:36.569537 11988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:03:36.569612 11988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:03:36.579243 11988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:33853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:33853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:36.580535 11988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:36.581918 11988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:36.594053 11997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:36.594142 11995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:36.594095 11994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:36.598553 11988 server_base.cc:1034] running on GCE node
I20260809 23:03:36.599363 11988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:36.600757 11988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:36.602104 11988 hybrid_clock.cc:648] HybridClock initialized: now 1786316616602060 us; error 52 us; skew 500 ppm
I20260809 23:03:36.602485 11988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:36 dist-test-slave-vxj2 krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316616, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:36.605677 11988 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:03:36.607336 11988 webserver.cc:460] Webserver started at http://127.24.31.254:35293/ using document root <none> and password file <none>
I20260809 23:03:36.608065 11988 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:36.608212 11988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:36.608525 11988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:36.611544 11988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "523d805846654f8ebc680a05cd4799fd"
format_stamp: "Formatted at 2026-08-09 23:03:36 on dist-test-slave-vxj2"
server_key: "0737a4fd5b26a7ec5b70b6de2926caac"
server_key_iv: "d3df9d9d37813f206d5bfd37ca7b8cdd"
server_key_version: "encryptionkey@0"
I20260809 23:03:36.612346 11988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "523d805846654f8ebc680a05cd4799fd"
format_stamp: "Formatted at 2026-08-09 23:03:36 on dist-test-slave-vxj2"
server_key: "0737a4fd5b26a7ec5b70b6de2926caac"
server_key_iv: "d3df9d9d37813f206d5bfd37ca7b8cdd"
server_key_version: "encryptionkey@0"
I20260809 23:03:36.617604 11988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 23:03:36.621579 12004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:36.622920 11988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 23:03:36.623116 11988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "523d805846654f8ebc680a05cd4799fd"
format_stamp: "Formatted at 2026-08-09 23:03:36 on dist-test-slave-vxj2"
server_key: "0737a4fd5b26a7ec5b70b6de2926caac"
server_key_iv: "d3df9d9d37813f206d5bfd37ca7b8cdd"
server_key_version: "encryptionkey@0"
I20260809 23:03:36.623322 11988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:36.636145 11988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:36.640166 11988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:36.640445 11988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:36.660028 11988 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:33853
I20260809 23:03:36.660058 12062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:33853 every 8 connection(s)
I20260809 23:03:36.661522 11988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:03:36.665850 12063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:36.670379 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 11988
I20260809 23:03:36.670603 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:03:36.671272 24703 external_mini_cluster.cc:1428] Setting key 2d1d8ed7710c8dc6715a9cf4030ce086
I20260809 23:03:36.680824 12063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd: Bootstrap starting.
I20260809 23:03:36.685451 12063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:36.687016 12063 log.cc:824] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd: Log is configured to *not* fsync() on all Append() calls
Aug 09 23:03:36 dist-test-slave-vxj2 krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:36.690831 12063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd: No bootstrap required, opened a new log
I20260809 23:03:36.697996 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:36.673318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.24.31.254:33853)
0809 23:03:36.674100 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:36.674111 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:36.674151 (+ 40us) server_negotiation.cc:407] Connection header received
0809 23:03:36.675782 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:36.675802 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:36.676178 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:36.676535 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:36.678871 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:36.680191 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:36.682515 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:36.683245 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:36.683390 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:36.689839 (+ 6449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:36.689847 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:36.689861 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:36.689902 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:36.693214 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:36.694250 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:36.694257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:36.694263 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:36.694458 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:36.694946 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:36.694949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:36.694951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:36.695514 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:36.695790 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:36.696311 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 23:03:36.696673 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":378,"threads_started":1}
I20260809 23:03:36.702209 12063 raft_consensus.cc:348] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523d805846654f8ebc680a05cd4799fd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33853 } }
I20260809 23:03:36.702682 12063 raft_consensus.cc:374] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:36.702844 12063 raft_consensus.cc:729] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 523d805846654f8ebc680a05cd4799fd, State: Initialized, Role: FOLLOWER
I20260809 23:03:36.703552 12063 consensus_queue.cc:260] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523d805846654f8ebc680a05cd4799fd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33853 } }
I20260809 23:03:36.703848 12063 raft_consensus.cc:388] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:03:36.704030 12063 raft_consensus.cc:482] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:03:36.704236 12063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:36.707278 12063 raft_consensus.cc:504] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523d805846654f8ebc680a05cd4799fd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33853 } }
I20260809 23:03:36.707856 12063 leader_election.cc:302] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 523d805846654f8ebc680a05cd4799fd; no voters:
I20260809 23:03:36.708531 12063 leader_election.cc:288] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:03:36.708629 12068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:36.709575 12068 raft_consensus.cc:686] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [term 1 LEADER]: Becoming Leader. State: Replica: 523d805846654f8ebc680a05cd4799fd, State: Running, Role: LEADER
I20260809 23:03:36.710292 12068 consensus_queue.cc:237] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523d805846654f8ebc680a05cd4799fd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33853 } }
I20260809 23:03:36.711335 12063 sys_catalog.cc:564] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:03:36.713518 12070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 523d805846654f8ebc680a05cd4799fd. Latest consensus state: current_term: 1 leader_uuid: "523d805846654f8ebc680a05cd4799fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523d805846654f8ebc680a05cd4799fd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33853 } } }
I20260809 23:03:36.714567 12070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:36.713825 12069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "523d805846654f8ebc680a05cd4799fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "523d805846654f8ebc680a05cd4799fd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33853 } } }
I20260809 23:03:36.715176 12069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:36.717983 12076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:03:36.724318 12076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:03:36.735683 12076 catalog_manager.cc:1347] Generated new cluster ID: 483aece9a3684152950c6a0527a5aaaf
I20260809 23:03:36.735862 12076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:03:36.761457 12076 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:03:36.762799 12076 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:03:36.776204 12076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd: Generated new TSK 0
I20260809 23:03:36.777053 12076 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 23:03:36.855167 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33853
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:37.191159 12090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:37.191419 12090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:37.191515 12090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:37.198729 12090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:37.198838 12090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:37.198951 12090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:03:37.208715 12090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:37.210007 12090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:37.211452 12090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:37.224795 12096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:37.224889 12095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:37.224828 12098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:37.226052 12090 server_base.cc:1034] running on GCE node
I20260809 23:03:37.226694 12090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:37.227732 12090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:37.228998 12090 hybrid_clock.cc:648] HybridClock initialized: now 1786316617228881 us; error 95 us; skew 500 ppm
I20260809 23:03:37.229405 12090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:37 dist-test-slave-vxj2 krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316617, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:37.232745 12090 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:03:37.234349 12090 webserver.cc:460] Webserver started at http://127.24.31.193:32883/ using document root <none> and password file <none>
I20260809 23:03:37.235044 12090 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:37.235203 12090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:37.235598 12090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:37.238651 12090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "98bec958537047c7984cc9aade2f13d9"
format_stamp: "Formatted at 2026-08-09 23:03:37 on dist-test-slave-vxj2"
server_key: "f652f86a6724410ad8adb73568827f23"
server_key_iv: "8b7f7cea655e09f025ef136e2ffa5ae7"
server_key_version: "encryptionkey@0"
I20260809 23:03:37.239497 12090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "98bec958537047c7984cc9aade2f13d9"
format_stamp: "Formatted at 2026-08-09 23:03:37 on dist-test-slave-vxj2"
server_key: "f652f86a6724410ad8adb73568827f23"
server_key_iv: "8b7f7cea655e09f025ef136e2ffa5ae7"
server_key_version: "encryptionkey@0"
I20260809 23:03:37.244863 12090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 23:03:37.248518 12106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:37.250100 12090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 23:03:37.250308 12090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "98bec958537047c7984cc9aade2f13d9"
format_stamp: "Formatted at 2026-08-09 23:03:37 on dist-test-slave-vxj2"
server_key: "f652f86a6724410ad8adb73568827f23"
server_key_iv: "8b7f7cea655e09f025ef136e2ffa5ae7"
server_key_version: "encryptionkey@0"
I20260809 23:03:37.250521 12090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:37.279562 12090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:37.283757 12090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:37.284087 12090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:37.285035 12090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:37.286455 12090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:37.286580 12090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:37.286731 12090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:37.286814 12090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:37.321206 12090 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:43657
I20260809 23:03:37.321328 12234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:43657 every 8 connection(s)
I20260809 23:03:37.322772 12090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:03:37.332530 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 12090
I20260809 23:03:37.332778 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:03:37.333559 24703 external_mini_cluster.cc:1428] Setting key dc78d2404d0e6b20f2879d1f42a85509
Aug 09 23:03:37 dist-test-slave-vxj2 krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316617, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:37.342861 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:37.325410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:36755 (local address 127.24.31.254:33853)
0809 23:03:37.325886 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:37.325891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:37.326702 (+ 811us) server_negotiation.cc:407] Connection header received
0809 23:03:37.328585 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:37.328590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:37.328681 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:37.328812 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:37.330354 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:37.331084 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:37.332655 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:37.333141 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:37.333352 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:37.337260 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:37.337265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:37.337268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:37.337299 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:37.339835 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:37.340550 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:37.340557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:37.340560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:37.340684 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:37.341127 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:37.341132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:37.341135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:37.341427 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:37.341581 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:37.342250 (+ 669us) server_negotiation.cc:299] Negotiation successful
0809 23:03:37.342479 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":239,"threads_started":1}
I20260809 23:03:37.343737 12238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:37.325833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:33853 (local address 127.24.31.193:36755)
0809 23:03:37.326546 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:37.326584 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:37.328327 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:37.329036 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:37.329043 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:37.329606 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:37.330152 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:37.330168 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:37.331248 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:37.331264 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:37.332501 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:37.332514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:37.333440 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:37.333449 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:37.333746 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:37.334443 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:03:37.334474 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:37.337018 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:37.339980 (+ 2962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:37.339997 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:37.340011 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:37.340430 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:37.340803 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:37.340808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:37.340812 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:37.341015 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:37.341556 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:37.341564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:37.341890 (+ 326us) client_negotiation.cc:769] Sending connection context
0809 23:03:37.342139 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 23:03:37.342660 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":278,"threads_started":1}
I20260809 23:03:37.345355 12235 heartbeater.cc:347] Connected to a master server at 127.24.31.254:33853
I20260809 23:03:37.345669 12235 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:37.346596 12235 heartbeater.cc:511] Master 127.24.31.254:33853 requested a full tablet report, sending...
I20260809 23:03:37.348809 12022 ts_manager.cc:194] Registered new tserver with Master: 98bec958537047c7984cc9aade2f13d9 (127.24.31.193:43657)
I20260809 23:03:37.350312 12022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:36755
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 23:03:37.398059 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33853
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:37.732573 12241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:37.732903 12241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:37.733088 12241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:37.740592 12241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:37.740765 12241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:37.740974 12241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:03:37.751116 12241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:37.752486 12241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:37.753970 12241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:37.768049 12247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:37.768245 12250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:37.768064 12248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:37.768940 12241 server_base.cc:1034] running on GCE node
I20260809 23:03:37.769557 12241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:37.770417 12241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:37.771646 12241 hybrid_clock.cc:648] HybridClock initialized: now 1786316617771582 us; error 47 us; skew 500 ppm
I20260809 23:03:37.772025 12241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:37 dist-test-slave-vxj2 krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316617, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:37.775355 12241 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:03:37.776840 12241 webserver.cc:460] Webserver started at http://127.24.31.194:44343/ using document root <none> and password file <none>
I20260809 23:03:37.777585 12241 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:37.777719 12241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:37.778046 12241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:37.780853 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "d165cfafce31413b9093f04cceb0f6e2"
format_stamp: "Formatted at 2026-08-09 23:03:37 on dist-test-slave-vxj2"
server_key: "7ae91f0de7c932569addbc499a02b488"
server_key_iv: "7ab7680ef880c2410c3ab9157cd41ed5"
server_key_version: "encryptionkey@0"
I20260809 23:03:37.781675 12241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "d165cfafce31413b9093f04cceb0f6e2"
format_stamp: "Formatted at 2026-08-09 23:03:37 on dist-test-slave-vxj2"
server_key: "7ae91f0de7c932569addbc499a02b488"
server_key_iv: "7ab7680ef880c2410c3ab9157cd41ed5"
server_key_version: "encryptionkey@0"
I20260809 23:03:37.786671 12241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 23:03:37.790009 12258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:37.791299 12241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:03:37.791474 12241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "d165cfafce31413b9093f04cceb0f6e2"
format_stamp: "Formatted at 2026-08-09 23:03:37 on dist-test-slave-vxj2"
server_key: "7ae91f0de7c932569addbc499a02b488"
server_key_iv: "7ab7680ef880c2410c3ab9157cd41ed5"
server_key_version: "encryptionkey@0"
I20260809 23:03:37.791666 12241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:37.825578 12241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:37.830166 12241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:37.830487 12241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:37.831444 12241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:37.832784 12241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:37.832909 12241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:37.833086 12241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:37.833202 12241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:37.868331 12241 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:37475
I20260809 23:03:37.868423 12390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:37475 every 8 connection(s)
I20260809 23:03:37.869655 12241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:03:37.875114 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 12241
I20260809 23:03:37.875312 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:03:37.875895 24703 external_mini_cluster.cc:1428] Setting key 50c33527cde3187cb0f79663b0289ea2
Aug 09 23:03:37 dist-test-slave-vxj2 krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316617, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:37.893596 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:37.872475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:56279 (local address 127.24.31.254:33853)
0809 23:03:37.872955 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:37.872959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:37.873768 (+ 809us) server_negotiation.cc:407] Connection header received
0809 23:03:37.876186 (+ 2418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:37.876190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:37.876263 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:37.876388 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:37.878218 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:37.878962 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:37.881460 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:37.882112 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:37.882258 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:37.886733 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:37.886742 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:37.886750 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:37.886798 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:37.890061 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:37.890988 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:37.890996 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:37.891003 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:37.891179 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:37.891581 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:37.891589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:37.891595 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:37.891993 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:37.892150 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:37.892812 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 23:03:37.893074 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":254,"threads_started":1}
I20260809 23:03:37.894214 12394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:37.872905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:33853 (local address 127.24.31.194:56279)
0809 23:03:37.873605 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:37.873649 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:37.875874 (+ 2225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:37.876656 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:37.876664 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:37.877360 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:37.878043 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:37.878070 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:37.879128 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:37.879136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:37.881309 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:37.881326 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:37.882363 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:37.882373 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:37.882695 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:37.883460 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:03:37.883501 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:37.886447 (+ 2946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:37.890208 (+ 3761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:37.890216 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:37.890231 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:37.890853 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:37.891288 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:37.891291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:37.891294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:37.891465 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:37.892127 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:37.892132 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:37.892446 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 23:03:37.892662 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 23:03:37.893029 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":311,"threads_started":1}
I20260809 23:03:37.895856 12391 heartbeater.cc:347] Connected to a master server at 127.24.31.254:33853
I20260809 23:03:37.896178 12391 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:37.896915 12391 heartbeater.cc:511] Master 127.24.31.254:33853 requested a full tablet report, sending...
I20260809 23:03:37.898488 12022 ts_manager.cc:194] Registered new tserver with Master: d165cfafce31413b9093f04cceb0f6e2 (127.24.31.194:37475)
I20260809 23:03:37.899798 12022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:56279
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 23:03:37.940903 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33853
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:38.277462 12399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:38.277746 12399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:38.277853 12399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:38.284860 12399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:38.284963 12399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:38.285074 12399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:03:38.294677 12399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:33853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:38.295955 12399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:38.297449 12399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:38.310696 12406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:38.310724 12408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:38.310827 12405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:38.311537 12399 server_base.cc:1034] running on GCE node
I20260809 23:03:38.312153 12399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:38.313118 12399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:38.314405 12399 hybrid_clock.cc:648] HybridClock initialized: now 1786316618314358 us; error 39 us; skew 500 ppm
I20260809 23:03:38.314738 12399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:38 dist-test-slave-vxj2 krb5kdc[11973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:38.317881 12399 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:03:38.319372 12399 webserver.cc:460] Webserver started at http://127.24.31.195:37321/ using document root <none> and password file <none>
I20260809 23:03:38.320082 12399 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:38.320214 12399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:38.320608 12399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:38.324594 12399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "eed322544dda457babdc9acc94fa05ca"
format_stamp: "Formatted at 2026-08-09 23:03:38 on dist-test-slave-vxj2"
server_key: "4e89334e353aca0bdca2088f56891ec5"
server_key_iv: "6527d9e6066e9fce5313116317859d42"
server_key_version: "encryptionkey@0"
I20260809 23:03:38.325608 12399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "eed322544dda457babdc9acc94fa05ca"
format_stamp: "Formatted at 2026-08-09 23:03:38 on dist-test-slave-vxj2"
server_key: "4e89334e353aca0bdca2088f56891ec5"
server_key_iv: "6527d9e6066e9fce5313116317859d42"
server_key_version: "encryptionkey@0"
I20260809 23:03:38.331243 12399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:03:38.334581 12417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:38.335841 12399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:03:38.336016 12399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "eed322544dda457babdc9acc94fa05ca"
format_stamp: "Formatted at 2026-08-09 23:03:38 on dist-test-slave-vxj2"
server_key: "4e89334e353aca0bdca2088f56891ec5"
server_key_iv: "6527d9e6066e9fce5313116317859d42"
server_key_version: "encryptionkey@0"
I20260809 23:03:38.336221 12399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:38.353199 12235 heartbeater.cc:503] Master 127.24.31.254:33853 was elected leader, sending a full tablet report...
I20260809 23:03:38.356523 12399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:38.361280 12399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:38.361569 12399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:38.362633 12399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:38.363960 12399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:38.364066 12399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:38.364218 12399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:38.364298 12399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:38.400490 12399 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:46653
I20260809 23:03:38.400548 12548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:46653 every 8 connection(s)
I20260809 23:03:38.401865 12399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:03:38.407959 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 12399
I20260809 23:03:38.408164 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:03:38.408654 24703 external_mini_cluster.cc:1428] Setting key 64a319641f10e021f68822a57ca334ef
Aug 09 23:03:38 dist-test-slave-vxj2 krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316618, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:38.422065 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.404431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:58815 (local address 127.24.31.254:33853)
0809 23:03:38.404945 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.404950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.405748 (+ 798us) server_negotiation.cc:407] Connection header received
0809 23:03:38.407579 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.407585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.407682 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.407811 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:38.409544 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.410376 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.412074 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.412516 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.412651 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.416065 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:38.416075 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:38.416078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:38.416109 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:38.418832 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:38.419531 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:38.419538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:38.419546 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:38.419658 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:38.420097 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:38.420102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:38.420107 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:38.420406 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:38.420581 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.421400 (+ 819us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.421640 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":266,"threads_started":1}
I20260809 23:03:38.423116 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.404807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:33853 (local address 127.24.31.195:58815)
0809 23:03:38.405584 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:38.405628 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:38.407318 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:38.408060 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:38.408072 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:38.408633 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:38.409347 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.409376 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.410540 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.410550 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.411872 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.411910 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.412759 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.412767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.412996 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.413754 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:03:38.413783 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:38.415864 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:38.418967 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:38.418976 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:38.418989 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:38.419405 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:38.419777 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:38.419782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:38.419785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:38.419959 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:38.420546 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:38.420553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:38.420980 (+ 427us) client_negotiation.cc:769] Sending connection context
0809 23:03:38.421287 (+ 307us) client_negotiation.cc:240] Negotiation successful
0809 23:03:38.421789 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":342,"threads_started":1}
I20260809 23:03:38.424500 12550 heartbeater.cc:347] Connected to a master server at 127.24.31.254:33853
I20260809 23:03:38.424826 12550 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:38.425601 12550 heartbeater.cc:511] Master 127.24.31.254:33853 requested a full tablet report, sending...
I20260809 23:03:38.427013 12022 ts_manager.cc:194] Registered new tserver with Master: eed322544dda457babdc9acc94fa05ca (127.24.31.195:46653)
I20260809 23:03:38.427814 12022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:58815
I20260809 23:03:38.436168 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:03:38.454879 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.441425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54432 (local address 127.24.31.254:33853)
0809 23:03:38.441662 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.441671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.442068 (+ 397us) server_negotiation.cc:407] Connection header received
0809 23:03:38.442207 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.442211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.442271 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.442351 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:38.443924 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.444653 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.446182 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.446601 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.446707 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.449410 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:38.449415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:38.449418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:38.449450 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:38.451913 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:38.452601 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:38.452605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:38.452607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:38.452728 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:38.453270 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:38.453276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:38.453279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:38.453615 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:38.453896 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.454300 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.454525 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 23:03:38.459381 12022 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:03:38.466161 12022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:03:38.500945 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.479363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:43657 (local address 127.0.0.1:55570)
0809 23:03:38.483259 (+ 3896us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:38.483278 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:38.483398 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:38.493142 (+ 9744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:38.493172 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:38.493205 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:38.493692 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.493705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.495658 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.495664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.497891 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.497905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.499836 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.499843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.500032 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.500088 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 23:03:38.500224 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 23:03:38.500423 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3759,"mutex_wait_us":43,"thread_start_us":1452,"threads_started":1}
I20260809 23:03:38.501236 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.479978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55570 (local address 127.24.31.193:43657)
0809 23:03:38.492371 (+ 12393us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.492386 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.492428 (+ 42us) server_negotiation.cc:407] Connection header received
0809 23:03:38.492580 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.492593 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.492895 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.493134 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:38.493957 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.495470 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.498081 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.499605 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.499775 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.500046 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.500494 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.500691 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12130,"thread_start_us":11931,"threads_started":1}
I20260809 23:03:38.501832 12569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.482363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:37475 (local address 127.0.0.1:35756)
0809 23:03:38.491248 (+ 8885us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:38.491273 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:38.491438 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:38.493270 (+ 1832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:38.493275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:38.493331 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:38.493903 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.493917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.496256 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.496260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.498271 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.498291 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.500234 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.500242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.500460 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.500497 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 23:03:38.500602 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 23:03:38.500732 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8706,"mutex_wait_us":1,"spinlock_wait_cycles":3840,"thread_start_us":9178,"threads_started":1}
I20260809 23:03:38.502179 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.484213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.24.31.194:37475)
0809 23:03:38.492621 (+ 8408us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.492632 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.492663 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:03:38.492770 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.492778 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.493002 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.493946 (+ 944us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:38.494165 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.496118 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.498467 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.499980 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.500143 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.500353 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.501038 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.501861 (+ 823us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8239,"thread_start_us":8109,"threads_started":1}
I20260809 23:03:38.505574 12318 tablet_service.cc:1467] Processing CreateTablet for tablet ba2039fb23004261aabbbb4629f717c5 (DEFAULT_TABLE table=test-table [id=c516bd65c3d44ebab1ba8b18c3c06e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:38.505584 12163 tablet_service.cc:1467] Processing CreateTablet for tablet ba2039fb23004261aabbbb4629f717c5 (DEFAULT_TABLE table=test-table [id=c516bd65c3d44ebab1ba8b18c3c06e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:38.506798 12318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba2039fb23004261aabbbb4629f717c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:38.506827 12163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba2039fb23004261aabbbb4629f717c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:38.509517 12566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.479340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:46653 (local address 127.0.0.1:52566)
0809 23:03:38.481360 (+ 2020us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:38.481422 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:38.481627 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:38.493657 (+ 12030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:38.493664 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:38.493690 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:38.494208 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.494235 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.496160 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.496168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.499031 (+ 2863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.499043 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.500836 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.500842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.501046 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.501308 (+ 262us) client_negotiation.cc:769] Sending connection context
0809 23:03:38.508909 (+ 7601us) client_negotiation.cc:240] Negotiation successful
0809 23:03:38.509067 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1627,"mutex_wait_us":6,"thread_start_us":2564,"threads_started":1}
I20260809 23:03:38.510273 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.479497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52566 (local address 127.24.31.195:46653)
0809 23:03:38.493057 (+ 13560us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.493064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.493093 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:03:38.493226 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.493232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.493436 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.493626 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:38.494422 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.495982 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.499190 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.500351 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.500466 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.500679 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.509750 (+ 9071us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.509945 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13384,"thread_start_us":13211,"threads_started":1}
I20260809 23:03:38.513808 12474 tablet_service.cc:1467] Processing CreateTablet for tablet ba2039fb23004261aabbbb4629f717c5 (DEFAULT_TABLE table=test-table [id=c516bd65c3d44ebab1ba8b18c3c06e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:38.515316 12474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba2039fb23004261aabbbb4629f717c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:38.518579 12571 tablet_bootstrap.cc:492] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: Bootstrap starting.
I20260809 23:03:38.521744 12571 tablet_bootstrap.cc:654] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:38.523121 12571 log.cc:824] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:38.523568 12572 tablet_bootstrap.cc:492] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: Bootstrap starting.
I20260809 23:03:38.526708 12571 tablet_bootstrap.cc:492] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: No bootstrap required, opened a new log
I20260809 23:03:38.527009 12571 ts_tablet_manager.cc:1397] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 23:03:38.527192 12572 tablet_bootstrap.cc:654] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:38.528522 12572 log.cc:824] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:38.533948 12574 tablet_bootstrap.cc:492] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: Bootstrap starting.
I20260809 23:03:38.536685 12572 tablet_bootstrap.cc:492] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: No bootstrap required, opened a new log
I20260809 23:03:38.536966 12572 ts_tablet_manager.cc:1397] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260809 23:03:38.537581 12574 tablet_bootstrap.cc:654] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:38.538717 12574 log.cc:824] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:38.538869 12571 raft_consensus.cc:348] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } }
I20260809 23:03:38.539311 12571 raft_consensus.cc:374] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:38.539428 12571 raft_consensus.cc:729] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d165cfafce31413b9093f04cceb0f6e2, State: Initialized, Role: FOLLOWER
I20260809 23:03:38.540087 12571 consensus_queue.cc:260] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } }
I20260809 23:03:38.541224 12391 heartbeater.cc:503] Master 127.24.31.254:33853 was elected leader, sending a full tablet report...
I20260809 23:03:38.541895 12571 ts_tablet_manager.cc:1428] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260809 23:03:38.542382 12574 tablet_bootstrap.cc:492] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: No bootstrap required, opened a new log
I20260809 23:03:38.542645 12574 ts_tablet_manager.cc:1397] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 23:03:38.548177 12572 raft_consensus.cc:348] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } }
I20260809 23:03:38.548521 12572 raft_consensus.cc:374] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:38.548627 12572 raft_consensus.cc:729] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98bec958537047c7984cc9aade2f13d9, State: Initialized, Role: FOLLOWER
I20260809 23:03:38.549229 12572 consensus_queue.cc:260] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } }
I20260809 23:03:38.550715 12572 ts_tablet_manager.cc:1428] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260809 23:03:38.550719 12574 raft_consensus.cc:348] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } }
I20260809 23:03:38.551617 12574 raft_consensus.cc:374] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:38.551733 12574 raft_consensus.cc:729] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eed322544dda457babdc9acc94fa05ca, State: Initialized, Role: FOLLOWER
I20260809 23:03:38.552280 12574 consensus_queue.cc:260] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } }
I20260809 23:03:38.553344 12550 heartbeater.cc:503] Master 127.24.31.254:33853 was elected leader, sending a full tablet report...
I20260809 23:03:38.553977 12574 ts_tablet_manager.cc:1428] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
W20260809 23:03:38.575227 12236 tablet.cc:2369] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:38.612679 12577 raft_consensus.cc:482] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:03:38.612982 12577 raft_consensus.cc:504] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } }
I20260809 23:03:38.614750 12577 leader_election.cc:288] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eed322544dda457babdc9acc94fa05ca (127.24.31.195:46653), 98bec958537047c7984cc9aade2f13d9 (127.24.31.193:43657)
I20260809 23:03:38.622044 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.615376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:49599 (local address 127.24.31.193:43657)
0809 23:03:38.615621 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.615626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.616340 (+ 714us) server_negotiation.cc:407] Connection header received
0809 23:03:38.616577 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.616581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.616686 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.616812 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:38.617502 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.618275 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.620132 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.620945 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.621105 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.621226 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.621605 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.621734 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 23:03:38.621989 12392 tablet.cc:2369] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:38.622174 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.615157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:55465 (local address 127.24.31.195:46653)
0809 23:03:38.615380 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.615389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.616194 (+ 805us) server_negotiation.cc:407] Connection header received
0809 23:03:38.616457 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.616464 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.616553 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.616621 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:38.617368 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.618165 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.620186 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.620970 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.621175 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.621268 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.621796 (+ 528us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.621919 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 23:03:38.622355 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.615197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:46653 (local address 127.24.31.194:55465)
0809 23:03:38.616053 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:38.616074 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:38.616218 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:38.616689 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:38.616694 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:38.616721 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:38.617198 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.617221 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.618330 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.618337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.620010 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.620022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.621415 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.621421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.621621 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.621662 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 23:03:38.621768 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 23:03:38.621887 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":91,"thread_start_us":658,"threads_started":1}
I20260809 23:03:38.622951 12581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.615222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:43657 (local address 127.24.31.194:49599)
0809 23:03:38.616187 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:38.616208 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:38.616398 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:38.616833 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:38.616839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:38.616868 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:38.617327 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.617342 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.618417 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.618420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.619979 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:38.619991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.621217 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.621224 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:38.621408 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.621432 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 23:03:38.621556 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 23:03:38.621687 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":23,"thread_start_us":345,"threads_started":1}
I20260809 23:03:38.623386 12499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2039fb23004261aabbbb4629f717c5" candidate_uuid: "d165cfafce31413b9093f04cceb0f6e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed322544dda457babdc9acc94fa05ca" is_pre_election: true
I20260809 23:03:38.623934 12499 raft_consensus.cc:2454] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d165cfafce31413b9093f04cceb0f6e2 in term 0.
I20260809 23:03:38.623975 12184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2039fb23004261aabbbb4629f717c5" candidate_uuid: "d165cfafce31413b9093f04cceb0f6e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98bec958537047c7984cc9aade2f13d9" is_pre_election: true
I20260809 23:03:38.624526 12184 raft_consensus.cc:2454] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d165cfafce31413b9093f04cceb0f6e2 in term 0.
I20260809 23:03:38.624576 12260 leader_election.cc:302] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d165cfafce31413b9093f04cceb0f6e2, eed322544dda457babdc9acc94fa05ca; no voters:
I20260809 23:03:38.624925 12577 raft_consensus.cc:2781] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:03:38.625108 12577 raft_consensus.cc:482] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:03:38.625268 12577 raft_consensus.cc:3037] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:38.627693 12577 raft_consensus.cc:504] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } }
I20260809 23:03:38.628433 12577 leader_election.cc:288] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [CANDIDATE]: Term 1 election: Requested vote from peers eed322544dda457babdc9acc94fa05ca (127.24.31.195:46653), 98bec958537047c7984cc9aade2f13d9 (127.24.31.193:43657)
I20260809 23:03:38.628822 12499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2039fb23004261aabbbb4629f717c5" candidate_uuid: "d165cfafce31413b9093f04cceb0f6e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eed322544dda457babdc9acc94fa05ca"
I20260809 23:03:38.628966 12184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba2039fb23004261aabbbb4629f717c5" candidate_uuid: "d165cfafce31413b9093f04cceb0f6e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98bec958537047c7984cc9aade2f13d9"
I20260809 23:03:38.629029 12499 raft_consensus.cc:3037] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:38.629201 12184 raft_consensus.cc:3037] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:38.632341 12499 raft_consensus.cc:2454] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d165cfafce31413b9093f04cceb0f6e2 in term 1.
I20260809 23:03:38.632339 12184 raft_consensus.cc:2454] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d165cfafce31413b9093f04cceb0f6e2 in term 1.
I20260809 23:03:38.632776 12261 leader_election.cc:302] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98bec958537047c7984cc9aade2f13d9, d165cfafce31413b9093f04cceb0f6e2; no voters:
I20260809 23:03:38.633090 12577 raft_consensus.cc:2781] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:38.633562 12577 raft_consensus.cc:686] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 1 LEADER]: Becoming Leader. State: Replica: d165cfafce31413b9093f04cceb0f6e2, State: Running, Role: LEADER
I20260809 23:03:38.634027 12577 consensus_queue.cc:237] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } }
I20260809 23:03:38.638214 12022 catalog_manager.cc:5495] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d165cfafce31413b9093f04cceb0f6e2 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "d165cfafce31413b9093f04cceb0f6e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d165cfafce31413b9093f04cceb0f6e2" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 37475 } health_report { overall_health: HEALTHY } } }
W20260809 23:03:38.653563 12551 tablet.cc:2369] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:38.681744 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.676310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.24.31.194:37475)
0809 23:03:38.676506 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.676510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.676637 (+ 127us) server_negotiation.cc:407] Connection header received
0809 23:03:38.676833 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.676837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.676925 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.677055 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:03:38.677511 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.678267 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.679601 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.680006 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.680094 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.680525 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:03:38.681113 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:03:38.681314 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.681404 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.681482 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 23:03:38.689785 12499 raft_consensus.cc:1261] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [term 1 FOLLOWER]: Refusing update from remote peer d165cfafce31413b9093f04cceb0f6e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:38.689857 12184 raft_consensus.cc:1261] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [term 1 FOLLOWER]: Refusing update from remote peer d165cfafce31413b9093f04cceb0f6e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:38.690786 12582 consensus_queue.cc:1035] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eed322544dda457babdc9acc94fa05ca" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:38.691144 12577 consensus_queue.cc:1035] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98bec958537047c7984cc9aade2f13d9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 43657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:38.700848 12585 mvcc.cc:205] Tried to move back new op lower bound from 7316752870143184896 to 7316752869928382464. Current Snapshot: MvccSnapshot[applied={T|T < 7316752870143184896}]
I20260809 23:03:38.714932 12589 mvcc.cc:205] Tried to move back new op lower bound from 7316752870143184896 to 7316752869928382464. Current Snapshot: MvccSnapshot[applied={T|T < 7316752870143184896}]
I20260809 23:03:38.720302 12022 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:38.720559 12022 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:38.724321 12022 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 523d805846654f8ebc680a05cd4799fd: Sending DeleteTablet for 3 replicas of tablet ba2039fb23004261aabbbb4629f717c5
I20260809 23:03:38.725561 12474 tablet_service.cc:1514] Processing DeleteTablet for tablet ba2039fb23004261aabbbb4629f717c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:38 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:52566
I20260809 23:03:38.725567 12163 tablet_service.cc:1514] Processing DeleteTablet for tablet ba2039fb23004261aabbbb4629f717c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:38 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:55570
I20260809 23:03:38.725930 12318 tablet_service.cc:1514] Processing DeleteTablet for tablet ba2039fb23004261aabbbb4629f717c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:38 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:35756
I20260809 23:03:38.728727 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 12090
I20260809 23:03:38.736956 12599 tablet_replica.cc:330] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: stopping tablet replica
I20260809 23:03:38.738026 12599 raft_consensus.cc:2229] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:03:38.739874 12599 raft_consensus.cc:2258] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:38.742862 12599 ts_tablet_manager.cc:1905] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:38.744078 12597 tablet_replica.cc:330] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: stopping tablet replica
I20260809 23:03:38.744531 12597 raft_consensus.cc:2229] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:38.744859 12596 tablet_replica.cc:330] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: stopping tablet replica
I20260809 23:03:38.745083 12597 raft_consensus.cc:2258] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:38.745383 12596 raft_consensus.cc:2229] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:38.745929 12596 raft_consensus.cc:2258] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:38.747010 12597 ts_tablet_manager.cc:1905] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 23:03:38 dist-test-slave-vxj2 krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 23:03:38.753525 12597 ts_tablet_manager.cc:1918] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:38.753734 12597 log.cc:1196] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/ba2039fb23004261aabbbb4629f717c5
I20260809 23:03:38.753850 12599 ts_tablet_manager.cc:1918] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:38.754046 12599 log.cc:1196] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/ba2039fb23004261aabbbb4629f717c5
I20260809 23:03:38.754377 12597 ts_tablet_manager.cc:1939] T ba2039fb23004261aabbbb4629f717c5 P eed322544dda457babdc9acc94fa05ca: Deleting consensus metadata
I20260809 23:03:38.754534 12596 ts_tablet_manager.cc:1905] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:38.754715 12599 ts_tablet_manager.cc:1939] T ba2039fb23004261aabbbb4629f717c5 P d165cfafce31413b9093f04cceb0f6e2: Deleting consensus metadata
I20260809 23:03:38.756398 12006 catalog_manager.cc:4841] TS eed322544dda457babdc9acc94fa05ca (127.24.31.195:46653): tablet ba2039fb23004261aabbbb4629f717c5 (table test-table [id=c516bd65c3d44ebab1ba8b18c3c06e27]) successfully deleted
I20260809 23:03:38.756769 12006 catalog_manager.cc:4841] TS d165cfafce31413b9093f04cceb0f6e2 (127.24.31.194:37475): tablet ba2039fb23004261aabbbb4629f717c5 (table test-table [id=c516bd65c3d44ebab1ba8b18c3c06e27]) successfully deleted
I20260809 23:03:38.761292 12596 ts_tablet_manager.cc:1918] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:38.761464 12596 log.cc:1196] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/ba2039fb23004261aabbbb4629f717c5
I20260809 23:03:38.762010 12596 ts_tablet_manager.cc:1939] T ba2039fb23004261aabbbb4629f717c5 P 98bec958537047c7984cc9aade2f13d9: Deleting consensus metadata
I20260809 23:03:38.763880 12008 catalog_manager.cc:4841] TS 98bec958537047c7984cc9aade2f13d9 (127.24.31.193:43657): tablet ba2039fb23004261aabbbb4629f717c5 (table test-table [id=c516bd65c3d44ebab1ba8b18c3c06e27]) successfully deleted
I20260809 23:03:38.766074 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.730467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.24.31.193:43657)
0809 23:03:38.730688 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.730697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.734929 (+ 4232us) server_negotiation.cc:407] Connection header received
0809 23:03:38.735149 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.735158 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.735272 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.735384 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:38.738901 (+ 3517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.740453 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.743103 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.744047 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.744202 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.758531 (+ 14329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:38.758546 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:38.758567 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:38.758631 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:38.762889 (+ 4258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:38.763870 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:38.763880 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:38.763883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:38.764058 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:38.764496 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:38.764500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:38.764502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:38.764838 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:38.765076 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.765601 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.765762 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 23:03:38.766803 12229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:38.856549 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 12090
I20260809 23:03:38.880329 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 12241
Aug 09 23:03:38 dist-test-slave-vxj2 krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 23:03:38.900889 12570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:38.881296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35774 (local address 127.24.31.194:37475)
0809 23:03:38.881475 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:38.881481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:38.881504 (+ 23us) server_negotiation.cc:407] Connection header received
0809 23:03:38.881583 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:38.881589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:38.881695 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:38.881836 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:38.883491 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.884806 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.886892 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:38.887574 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:38.887709 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:38.893531 (+ 5822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:38.893544 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:38.893562 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:38.893614 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:38.897732 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:38.898607 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:38.898616 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:38.898620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:38.898782 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:38.899214 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:38.899219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:38.899223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:38.899643 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:38.899984 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:38.900277 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 23:03:38.900502 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 23:03:38.901669 12383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:39.016224 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 12241
I20260809 23:03:39.038143 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 12399
Aug 09 23:03:39 dist-test-slave-vxj2 krb5kdc[11973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 23:03:39.053454 12567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:39.038904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.24.31.195:46653)
0809 23:03:39.039148 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:39.039153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:39.039191 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:03:39.039389 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:39.039399 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:39.039476 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:39.039569 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:39.040978 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:39.041774 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:39.043061 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:39.043453 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:39.043547 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:39.048027 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:39.048041 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:39.048057 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:39.048090 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:39.050963 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:39.051573 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:39.051581 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:39.051583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:39.051722 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:39.052067 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:39.052070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:39.052072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:39.052418 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:39.052634 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:39.052887 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 23:03:39.053048 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 23:03:39.054117 12544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:39.143508 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 12399
I20260809 23:03:39.162597 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 11988
I20260809 23:03:39.163764 12055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:39.277117 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 11988
2026-08-09T23:03:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4544 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T23:03:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:03:39Z Disabled control of system clock
I20260809 23:03:39.365695 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44517
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:44517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:03:39.715444 12613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:39.715776 12613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:39.715868 12613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:39.723295 12613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:03:39.723414 12613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:39.723457 12613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:39.723522 12613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:03:39.723572 12613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:03:39.733028 12613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:44517
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:44517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:39.734355 12613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:39.736155 12613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:39.748696 12620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:39.748787 12618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:39.748782 12622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:39.749863 12613 server_base.cc:1034] running on GCE node
I20260809 23:03:39.750619 12613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:39.752058 12613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:39.753298 12613 hybrid_clock.cc:648] HybridClock initialized: now 1786316619753237 us; error 75 us; skew 500 ppm
I20260809 23:03:39.753671 12613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:39.756445 12613 webserver.cc:460] Webserver started at http://127.24.31.254:43853/ using document root <none> and password file <none>
I20260809 23:03:39.757143 12613 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:39.757346 12613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:39.757673 12613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:39.760728 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "80e7691817b44f1fa871280beccf95cd"
format_stamp: "Formatted at 2026-08-09 23:03:39 on dist-test-slave-vxj2"
server_key: "75413e92b2e8589a34a01c68406fc8c6"
server_key_iv: "445c5c128f2a59f3051e14ef6ef1312d"
server_key_version: "encryptionkey@0"
I20260809 23:03:39.761617 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "80e7691817b44f1fa871280beccf95cd"
format_stamp: "Formatted at 2026-08-09 23:03:39 on dist-test-slave-vxj2"
server_key: "75413e92b2e8589a34a01c68406fc8c6"
server_key_iv: "445c5c128f2a59f3051e14ef6ef1312d"
server_key_version: "encryptionkey@0"
I20260809 23:03:39.766834 12613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 23:03:39.770589 12630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:39.771970 12613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 23:03:39.772181 12613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "80e7691817b44f1fa871280beccf95cd"
format_stamp: "Formatted at 2026-08-09 23:03:39 on dist-test-slave-vxj2"
server_key: "75413e92b2e8589a34a01c68406fc8c6"
server_key_iv: "445c5c128f2a59f3051e14ef6ef1312d"
server_key_version: "encryptionkey@0"
I20260809 23:03:39.772400 12613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:39.787940 12613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:39.788780 12613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:39.789070 12613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:39.808524 12613 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:44517
I20260809 23:03:39.808544 12689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:44517 every 8 connection(s)
I20260809 23:03:39.809900 12613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:03:39.811089 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 12613
I20260809 23:03:39.811293 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:03:39.811913 24703 external_mini_cluster.cc:1428] Setting key 5f6b14b898c272b01e8a36426a45e2ec
I20260809 23:03:39.814667 12690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:39.826040 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:39.813903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42432 (local address 127.24.31.254:44517)
0809 23:03:39.814971 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:39.814982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:39.815028 (+ 46us) server_negotiation.cc:407] Connection header received
0809 23:03:39.816718 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:39.816745 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:39.817248 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:39.817589 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:39.818207 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:39.819566 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:39.821552 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:39.822233 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:39.822367 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:39.823639 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:39.823647 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:39.823663 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:39.824209 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:39.824294 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:39.824327 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:39.824414 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:39.824666 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 23:03:39.825037 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":388,"threads_started":1}
I20260809 23:03:39.827528 12690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd: Bootstrap starting.
I20260809 23:03:39.830807 12690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:39.831789 12690 log.cc:824] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:39.834652 12690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd: No bootstrap required, opened a new log
I20260809 23:03:39.842056 12690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e7691817b44f1fa871280beccf95cd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44517 } }
I20260809 23:03:39.842474 12690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:39.842592 12690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e7691817b44f1fa871280beccf95cd, State: Initialized, Role: FOLLOWER
I20260809 23:03:39.843225 12690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e7691817b44f1fa871280beccf95cd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44517 } }
I20260809 23:03:39.843474 12690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:03:39.843614 12690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:03:39.843811 12690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:39.846195 12690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e7691817b44f1fa871280beccf95cd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44517 } }
I20260809 23:03:39.846712 12690 leader_election.cc:302] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80e7691817b44f1fa871280beccf95cd; no voters:
I20260809 23:03:39.847326 12690 leader_election.cc:288] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:03:39.847581 12696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:39.848403 12696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [term 1 LEADER]: Becoming Leader. State: Replica: 80e7691817b44f1fa871280beccf95cd, State: Running, Role: LEADER
I20260809 23:03:39.848892 12696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e7691817b44f1fa871280beccf95cd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44517 } }
I20260809 23:03:39.849556 12690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:03:39.851739 12697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80e7691817b44f1fa871280beccf95cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e7691817b44f1fa871280beccf95cd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44517 } } }
I20260809 23:03:39.852240 12698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80e7691817b44f1fa871280beccf95cd. Latest consensus state: current_term: 1 leader_uuid: "80e7691817b44f1fa871280beccf95cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e7691817b44f1fa871280beccf95cd" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 44517 } } }
I20260809 23:03:39.852459 12697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:39.852540 12698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:39.855198 12704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:03:39.860796 12704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:03:39.875630 12704 catalog_manager.cc:1347] Generated new cluster ID: 35c1e125700047349976aab81b0a8985
I20260809 23:03:39.875751 12704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:03:39.898399 12704 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:03:39.899714 12704 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:03:39.908245 12704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd: Generated new TSK 0
I20260809 23:03:39.909273 12704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:03:39.925351 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44517
--builtin_ntp_servers=127.24.31.212:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:03:40.282294 12715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:40.282578 12715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:40.282747 12715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:40.290092 12715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:40.290184 12715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:40.290388 12715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:03:40.299607 12715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:40.300936 12715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:40.302767 12715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:40.315891 12721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:40.316012 12723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:40.316589 12725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:40.316774 12715 server_base.cc:1034] running on GCE node
I20260809 23:03:40.317498 12715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:40.318487 12715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:40.319721 12715 hybrid_clock.cc:648] HybridClock initialized: now 1786316620319669 us; error 39 us; skew 500 ppm
I20260809 23:03:40.320122 12715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:40.322657 12715 webserver.cc:460] Webserver started at http://127.24.31.193:43473/ using document root <none> and password file <none>
I20260809 23:03:40.323477 12715 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:40.323611 12715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:40.323969 12715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:40.328457 12715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "641872e41e6f4983859b5a87475500ca"
format_stamp: "Formatted at 2026-08-09 23:03:40 on dist-test-slave-vxj2"
server_key: "bd5ed3916120ce40092d78485f656e80"
server_key_iv: "8cf04f04689569fac2c3751dca36d718"
server_key_version: "encryptionkey@0"
I20260809 23:03:40.329442 12715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "641872e41e6f4983859b5a87475500ca"
format_stamp: "Formatted at 2026-08-09 23:03:40 on dist-test-slave-vxj2"
server_key: "bd5ed3916120ce40092d78485f656e80"
server_key_iv: "8cf04f04689569fac2c3751dca36d718"
server_key_version: "encryptionkey@0"
I20260809 23:03:40.334986 12715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:03:40.338622 12732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:40.340076 12715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:03:40.340255 12715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "641872e41e6f4983859b5a87475500ca"
format_stamp: "Formatted at 2026-08-09 23:03:40 on dist-test-slave-vxj2"
server_key: "bd5ed3916120ce40092d78485f656e80"
server_key_iv: "8cf04f04689569fac2c3751dca36d718"
server_key_version: "encryptionkey@0"
I20260809 23:03:40.340509 12715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:40.362404 12715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:40.363265 12715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:40.363590 12715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:40.364583 12715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:40.366179 12715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:40.366307 12715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:40.366442 12715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:40.366559 12715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:40.401352 12715 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:34453
I20260809 23:03:40.401445 12867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:34453 every 8 connection(s)
I20260809 23:03:40.402679 12715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:03:40.405932 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 12715
I20260809 23:03:40.406145 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:03:40.406848 24703 external_mini_cluster.cc:1428] Setting key 9774f9bb4b0ae46a23075262754f44aa
I20260809 23:03:40.410180 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44517
--builtin_ntp_servers=127.24.31.212:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:03:40.417493 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:40.405370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:56441 (local address 127.24.31.254:44517)
0809 23:03:40.405918 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:40.405926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:40.407401 (+ 1475us) server_negotiation.cc:407] Connection header received
0809 23:03:40.409821 (+ 2420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:40.409826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:40.409888 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:40.409983 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:40.411406 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:40.412118 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:40.414537 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:40.414956 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:40.415050 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:40.416677 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:40.416681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:40.416684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:40.416701 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:40.416739 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:40.416751 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:40.416837 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:40.417096 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 23:03:40.417230 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":273,"threads_started":1}
I20260809 23:03:40.418908 12872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:40.405996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44517 (local address 127.24.31.193:56441)
0809 23:03:40.407211 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:40.407259 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:40.409510 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:40.410253 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:40.410265 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:40.410398 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:40.411213 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:40.411242 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:40.412279 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:40.412288 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:40.414399 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:40.414417 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:40.415180 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:40.415189 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:40.415549 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:40.416306 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:03:40.416312 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:40.416365 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:03:40.416471 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:40.416874 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:40.416888 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:40.416899 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 23:03:40.417077 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 23:03:40.417527 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":348,"threads_started":1}
I20260809 23:03:40.420459 12869 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44517
I20260809 23:03:40.420755 12869 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:40.421681 12869 heartbeater.cc:511] Master 127.24.31.254:44517 requested a full tablet report, sending...
I20260809 23:03:40.423779 12653 ts_manager.cc:194] Registered new tserver with Master: 641872e41e6f4983859b5a87475500ca (127.24.31.193:34453)
I20260809 23:03:40.425355 12653 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.193:56441
W20260809 23:03:40.746659 12873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:40.746982 12873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:40.747123 12873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:40.754592 12873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:40.754688 12873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:40.754850 12873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:03:40.764777 12873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:40.766106 12873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:40.767935 12873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:40.781085 12880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:40.781132 12881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:40.781038 12883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:40.781836 12873 server_base.cc:1034] running on GCE node
I20260809 23:03:40.782436 12873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:40.783353 12873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:40.784582 12873 hybrid_clock.cc:648] HybridClock initialized: now 1786316620784532 us; error 61 us; skew 500 ppm
I20260809 23:03:40.784889 12873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:40.787079 12873 webserver.cc:460] Webserver started at http://127.24.31.194:41551/ using document root <none> and password file <none>
I20260809 23:03:40.787770 12873 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:40.787907 12873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:40.788218 12873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:40.791054 12873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "13b5e38fc7964402a5cde866ca7bae65"
format_stamp: "Formatted at 2026-08-09 23:03:40 on dist-test-slave-vxj2"
server_key: "29bab9e9a0d971f8f05dba5ab41b4a01"
server_key_iv: "fd98d08174fe932f540895d5d513bd16"
server_key_version: "encryptionkey@0"
I20260809 23:03:40.791872 12873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "13b5e38fc7964402a5cde866ca7bae65"
format_stamp: "Formatted at 2026-08-09 23:03:40 on dist-test-slave-vxj2"
server_key: "29bab9e9a0d971f8f05dba5ab41b4a01"
server_key_iv: "fd98d08174fe932f540895d5d513bd16"
server_key_version: "encryptionkey@0"
I20260809 23:03:40.796933 12873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260809 23:03:40.800246 12891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:40.801518 12873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:03:40.801693 12873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "13b5e38fc7964402a5cde866ca7bae65"
format_stamp: "Formatted at 2026-08-09 23:03:40 on dist-test-slave-vxj2"
server_key: "29bab9e9a0d971f8f05dba5ab41b4a01"
server_key_iv: "fd98d08174fe932f540895d5d513bd16"
server_key_version: "encryptionkey@0"
I20260809 23:03:40.801916 12873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:40.817595 12873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:40.818394 12873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:40.818696 12873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:40.819641 12873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:40.821146 12873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:40.821336 12873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 23:03:40.821465 12873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:40.821560 12873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:40.853406 12873 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:39627
I20260809 23:03:40.853446 13017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:39627 every 8 connection(s)
I20260809 23:03:40.854719 12873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:03:40.861496 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 12873
I20260809 23:03:40.861685 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:03:40.862293 24703 external_mini_cluster.cc:1428] Setting key 039093c38af35bd2da7790709e31602b
I20260809 23:03:40.865454 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44517
--builtin_ntp_servers=127.24.31.212:43095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:03:40.867761 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:40.857257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:41819 (local address 127.24.31.254:44517)
0809 23:03:40.857470 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:40.857476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:40.858846 (+ 1370us) server_negotiation.cc:407] Connection header received
0809 23:03:40.860508 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:40.860512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:40.860578 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:40.860670 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:40.861813 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:40.862905 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:40.864641 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:40.865302 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:40.865434 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:40.866821 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:40.866831 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:40.866839 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:40.866866 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:40.866937 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:40.866957 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:40.867055 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:40.867252 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 23:03:40.867356 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 23:03:40.868536 13021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:40.857809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44517 (local address 127.24.31.194:41819)
0809 23:03:40.858686 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:40.858727 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:40.860268 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:40.860860 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:40.860867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:40.860945 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:40.861625 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:40.861652 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:40.863077 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:40.863082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:40.864499 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:40.864511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:40.865493 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:40.865498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:40.865801 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:40.866515 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:03:40.866518 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:40.866554 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:03:40.866623 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:40.867059 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:40.867068 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:40.867073 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:03:40.867207 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 23:03:40.867484 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":269,"threads_started":1}
I20260809 23:03:40.869939 13018 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44517
I20260809 23:03:40.870234 13018 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:40.870946 13018 heartbeater.cc:511] Master 127.24.31.254:44517 requested a full tablet report, sending...
I20260809 23:03:40.872378 12653 ts_manager.cc:194] Registered new tserver with Master: 13b5e38fc7964402a5cde866ca7bae65 (127.24.31.194:39627)
I20260809 23:03:40.873351 12653 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.194:41819
W20260809 23:03:41.199220 13022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:41.199481 13022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:41.199568 13022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:41.206979 13022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:41.207074 13022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:41.207177 13022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:03:41.216064 13022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:44517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:41.217286 13022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:41.219044 13022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:41.232298 13030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:41.232335 13027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:41.232378 13028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:41.233433 13022 server_base.cc:1034] running on GCE node
I20260809 23:03:41.234043 13022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:41.234934 13022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:41.236146 13022 hybrid_clock.cc:648] HybridClock initialized: now 1786316621236113 us; error 24 us; skew 500 ppm
I20260809 23:03:41.236467 13022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:41.238879 13022 webserver.cc:460] Webserver started at http://127.24.31.195:46853/ using document root <none> and password file <none>
I20260809 23:03:41.239542 13022 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:41.239657 13022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:41.239979 13022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:41.242795 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3"
format_stamp: "Formatted at 2026-08-09 23:03:41 on dist-test-slave-vxj2"
server_key: "9a1455a0eff0c8669c8a7ed10c70002d"
server_key_iv: "6ed990ccc74b5107b22b6fd95b74ec67"
server_key_version: "encryptionkey@0"
I20260809 23:03:41.243716 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3"
format_stamp: "Formatted at 2026-08-09 23:03:41 on dist-test-slave-vxj2"
server_key: "9a1455a0eff0c8669c8a7ed10c70002d"
server_key_iv: "6ed990ccc74b5107b22b6fd95b74ec67"
server_key_version: "encryptionkey@0"
I20260809 23:03:41.249373 13022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 23:03:41.252810 13037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:41.254312 13022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:03:41.254587 13022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3"
format_stamp: "Formatted at 2026-08-09 23:03:41 on dist-test-slave-vxj2"
server_key: "9a1455a0eff0c8669c8a7ed10c70002d"
server_key_iv: "6ed990ccc74b5107b22b6fd95b74ec67"
server_key_version: "encryptionkey@0"
I20260809 23:03:41.254839 13022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:41.280123 13022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:41.281095 13022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:41.281452 13022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:41.282461 13022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:41.284020 13022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:41.284139 13022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:41.284288 13022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:41.284385 13022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:41.318029 13022 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:41427
I20260809 23:03:41.318120 13167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:41427 every 8 connection(s)
I20260809 23:03:41.319363 13022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:03:41.324856 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 13022
I20260809 23:03:41.325103 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:03:41.325700 24703 external_mini_cluster.cc:1428] Setting key b03e7f8ac5dae24cb6a054fb265a2a07
I20260809 23:03:41.332737 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.322060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:37321 (local address 127.24.31.254:44517)
0809 23:03:41.322242 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.322247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.323593 (+ 1346us) server_negotiation.cc:407] Connection header received
0809 23:03:41.325527 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.325533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.325641 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.325757 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:41.327038 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.327943 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.329740 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.330143 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.330320 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.331769 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:41.331782 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:41.331786 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:41.331804 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:41.331836 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:41.331847 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:41.331990 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.332224 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.332335 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 23:03:41.333738 13170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.322494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:44517 (local address 127.24.31.195:37321)
0809 23:03:41.323415 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.323462 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:41.325244 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:41.325985 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:41.325994 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:41.326084 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:41.326800 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.326847 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.328101 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.328108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.329593 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.329606 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.330414 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.330422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.330689 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.331415 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:03:41.331420 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:41.331459 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:03:41.331537 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:41.331991 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:41.332005 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:41.332027 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 23:03:41.332212 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 23:03:41.332642 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":323,"threads_started":1}
I20260809 23:03:41.335116 13168 heartbeater.cc:347] Connected to a master server at 127.24.31.254:44517
I20260809 23:03:41.335471 13168 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:41.336261 13168 heartbeater.cc:511] Master 127.24.31.254:44517 requested a full tablet report, sending...
I20260809 23:03:41.337790 12653 ts_manager.cc:194] Registered new tserver with Master: fc22d9fe3f8246febe0d7cad5e6ec9f3 (127.24.31.195:41427)
I20260809 23:03:41.338593 12653 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.195:37321
I20260809 23:03:41.341007 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:03:41.351065 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.345202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.24.31.254:44517)
0809 23:03:41.345384 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.345388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.345873 (+ 485us) server_negotiation.cc:407] Connection header received
0809 23:03:41.346097 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.346101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.346178 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.346261 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:41.346734 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.347435 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.348817 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.349288 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.349392 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.350510 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:41.350515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:41.350518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:41.350535 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:41.350564 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:41.350576 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:41.350648 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.350778 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.350847 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:03:41.354882 12653 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:03:41.362058 12653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:03:41.402268 13182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.374009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34453 (local address 127.0.0.1:34630)
0809 23:03:41.389724 (+ 15715us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.389744 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:41.391320 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:41.393202 (+ 1882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:41.393215 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:41.393276 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:41.393837 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.393853 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.396500 (+ 2647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.396505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.398989 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.399004 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.401380 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.401386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.401591 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.401643 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 23:03:41.401761 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 23:03:41.401937 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15573,"thread_start_us":16959,"threads_started":1}
I20260809 23:03:41.403767 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.374247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.24.31.193:34453)
0809 23:03:41.391963 (+ 17716us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.391975 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.392013 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:03:41.392164 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.392175 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.392607 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.392849 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:41.394120 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.396329 (+ 2209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.399214 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.400813 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.400954 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.401255 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.403143 (+ 1888us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.403351 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17484,"thread_start_us":17329,"threads_started":1}
I20260809 23:03:41.403977 13186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.392089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:41427 (local address 127.0.0.1:49566)
0809 23:03:41.395620 (+ 3531us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.395641 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:41.395768 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:41.396497 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:41.396502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:41.396527 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:41.397045 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.397063 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.398770 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.398777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.400985 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.401000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.403157 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.403163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.403329 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.403358 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 23:03:41.403446 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 23:03:41.403598 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3407,"mutex_wait_us":1,"thread_start_us":3303,"threads_started":1}
I20260809 23:03:41.404983 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.390464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.24.31.195:41427)
0809 23:03:41.395375 (+ 4911us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.395385 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.395896 (+ 511us) server_negotiation.cc:407] Connection header received
0809 23:03:41.396023 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.396030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.396261 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.397305 (+ 1044us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:41.397620 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.398622 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.401230 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.402667 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.402794 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.403023 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.404323 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.404490 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4730,"thread_start_us":4556,"threads_started":1}
I20260809 23:03:41.405500 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.373730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:39627 (local address 127.0.0.1:59870)
0809 23:03:41.389589 (+ 15859us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.389626 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:41.389831 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:41.396790 (+ 6959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:41.396794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:41.396823 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:41.397377 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.397393 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.399248 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.399251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.400815 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.400833 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.404469 (+ 3636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.404475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.404649 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.404679 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 23:03:41.404764 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 23:03:41.404902 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15569,"mutex_wait_us":6,"thread_start_us":15395,"threads_started":1}
I20260809 23:03:41.406666 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.374104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59870 (local address 127.24.31.194:39627)
0809 23:03:41.396030 (+ 21926us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.396038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.396063 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:03:41.396159 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.396166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.396340 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.396529 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:41.397580 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.399080 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.401062 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.402632 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.402749 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.405952 (+ 3203us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.406027 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.406288 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21635,"thread_start_us":21465,"threads_started":1}
I20260809 23:03:41.408023 12790 tablet_service.cc:1467] Processing CreateTablet for tablet 0bcd8224f95a47e9ae134c24dac99fe2 (DEFAULT_TABLE table=test-table [id=f477d18c7bab4fa28d71be53fc92703e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:41.408567 13092 tablet_service.cc:1467] Processing CreateTablet for tablet 0bcd8224f95a47e9ae134c24dac99fe2 (DEFAULT_TABLE table=test-table [id=f477d18c7bab4fa28d71be53fc92703e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:41.409615 12790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bcd8224f95a47e9ae134c24dac99fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:41.409981 13092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bcd8224f95a47e9ae134c24dac99fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:41.409914 12944 tablet_service.cc:1467] Processing CreateTablet for tablet 0bcd8224f95a47e9ae134c24dac99fe2 (DEFAULT_TABLE table=test-table [id=f477d18c7bab4fa28d71be53fc92703e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:41.410936 12944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bcd8224f95a47e9ae134c24dac99fe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:41.421329 13187 tablet_bootstrap.cc:492] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: Bootstrap starting.
I20260809 23:03:41.425274 13187 tablet_bootstrap.cc:654] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:41.426692 13187 log.cc:824] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:41.427103 13188 tablet_bootstrap.cc:492] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca: Bootstrap starting.
I20260809 23:03:41.427251 13189 tablet_bootstrap.cc:492] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: Bootstrap starting.
I20260809 23:03:41.428877 12869 heartbeater.cc:503] Master 127.24.31.254:44517 was elected leader, sending a full tablet report...
I20260809 23:03:41.430733 13189 tablet_bootstrap.cc:654] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:41.430843 13187 tablet_bootstrap.cc:492] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: No bootstrap required, opened a new log
I20260809 23:03:41.430830 13188 tablet_bootstrap.cc:654] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:41.431103 13187 ts_tablet_manager.cc:1397] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 23:03:41.431938 13189 log.cc:824] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:41.431989 13188 log.cc:824] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:41.436573 13189 tablet_bootstrap.cc:492] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: No bootstrap required, opened a new log
I20260809 23:03:41.436586 13188 tablet_bootstrap.cc:492] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca: No bootstrap required, opened a new log
I20260809 23:03:41.436830 13189 ts_tablet_manager.cc:1397] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260809 23:03:41.436852 13188 ts_tablet_manager.cc:1397] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260809 23:03:41.442066 13187 raft_consensus.cc:348] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } }
I20260809 23:03:41.442549 13187 raft_consensus.cc:374] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:41.442651 13187 raft_consensus.cc:729] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b5e38fc7964402a5cde866ca7bae65, State: Initialized, Role: FOLLOWER
I20260809 23:03:41.443272 13187 consensus_queue.cc:260] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } }
I20260809 23:03:41.444442 13018 heartbeater.cc:503] Master 127.24.31.254:44517 was elected leader, sending a full tablet report...
I20260809 23:03:41.444221 13188 raft_consensus.cc:348] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } }
I20260809 23:03:41.444763 13188 raft_consensus.cc:374] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:41.444864 13188 raft_consensus.cc:729] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 641872e41e6f4983859b5a87475500ca, State: Initialized, Role: FOLLOWER
I20260809 23:03:41.445240 13187 ts_tablet_manager.cc:1428] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260809 23:03:41.445550 13188 consensus_queue.cc:260] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } }
I20260809 23:03:41.447031 13189 raft_consensus.cc:348] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } }
I20260809 23:03:41.447306 13188 ts_tablet_manager.cc:1428] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260809 23:03:41.447417 13189 raft_consensus.cc:374] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:41.447527 13189 raft_consensus.cc:729] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc22d9fe3f8246febe0d7cad5e6ec9f3, State: Initialized, Role: FOLLOWER
I20260809 23:03:41.448135 13189 consensus_queue.cc:260] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } }
I20260809 23:03:41.449306 13168 heartbeater.cc:503] Master 127.24.31.254:44517 was elected leader, sending a full tablet report...
I20260809 23:03:41.450001 13189 ts_tablet_manager.cc:1428] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: Time spent starting tablet: real 0.013s user 0.009s sys 0.006s
I20260809 23:03:41.465688 13194 raft_consensus.cc:482] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:03:41.465978 13194 raft_consensus.cc:504] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } }
I20260809 23:03:41.467749 13194 leader_election.cc:288] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13b5e38fc7964402a5cde866ca7bae65 (127.24.31.194:39627), fc22d9fe3f8246febe0d7cad5e6ec9f3 (127.24.31.195:41427)
I20260809 23:03:41.475862 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.468138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:58409 (local address 127.24.31.194:39627)
0809 23:03:41.468348 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.468353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.469028 (+ 675us) server_negotiation.cc:407] Connection header received
0809 23:03:41.469353 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.469361 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.469462 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.469523 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:41.470465 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.471410 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.473939 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.474832 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.475048 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.475131 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.475426 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.475565 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 23:03:41.475930 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.469099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:50233 (local address 127.24.31.195:41427)
0809 23:03:41.469353 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.469361 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.469769 (+ 408us) server_negotiation.cc:407] Connection header received
0809 23:03:41.469939 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.469946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.470030 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.470142 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:41.470752 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.471762 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.473974 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.474937 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.475121 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.475203 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.475504 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.475640 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260809 23:03:41.476074 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.468146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:39627 (local address 127.24.31.193:58409)
0809 23:03:41.468878 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.468896 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:41.469095 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:41.469734 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:41.469741 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:41.469815 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:41.470286 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.470313 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.471578 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.471584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.473682 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.473699 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.475042 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.475048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.475244 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.475270 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 23:03:41.475382 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 23:03:41.475517 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":22,"spinlock_wait_cycles":5504,"thread_start_us":376,"threads_started":1}
I20260809 23:03:41.476529 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.468973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:41427 (local address 127.24.31.193:50233)
0809 23:03:41.469648 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.469663 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:41.469801 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:41.470161 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:41.470164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:41.470187 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:41.470550 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.470558 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.471957 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.471965 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.473746 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:41.473761 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.475157 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.475163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:41.475333 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.475357 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 23:03:41.475475 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 23:03:41.475677 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":19,"spinlock_wait_cycles":5504,"thread_start_us":370,"threads_started":1}
I20260809 23:03:41.477339 12970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcd8224f95a47e9ae134c24dac99fe2" candidate_uuid: "641872e41e6f4983859b5a87475500ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b5e38fc7964402a5cde866ca7bae65" is_pre_election: true
I20260809 23:03:41.477721 13117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcd8224f95a47e9ae134c24dac99fe2" candidate_uuid: "641872e41e6f4983859b5a87475500ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" is_pre_election: true
I20260809 23:03:41.477937 12970 raft_consensus.cc:2454] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641872e41e6f4983859b5a87475500ca in term 0.
I20260809 23:03:41.478367 13117 raft_consensus.cc:2454] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 641872e41e6f4983859b5a87475500ca in term 0.
I20260809 23:03:41.478745 12735 leader_election.cc:302] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13b5e38fc7964402a5cde866ca7bae65, 641872e41e6f4983859b5a87475500ca; no voters:
I20260809 23:03:41.479249 13194 raft_consensus.cc:2781] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:03:41.479434 13194 raft_consensus.cc:482] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:03:41.479564 13194 raft_consensus.cc:3037] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:41.482218 13194 raft_consensus.cc:504] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } }
I20260809 23:03:41.482877 13194 leader_election.cc:288] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [CANDIDATE]: Term 1 election: Requested vote from peers 13b5e38fc7964402a5cde866ca7bae65 (127.24.31.194:39627), fc22d9fe3f8246febe0d7cad5e6ec9f3 (127.24.31.195:41427)
I20260809 23:03:41.483362 12970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcd8224f95a47e9ae134c24dac99fe2" candidate_uuid: "641872e41e6f4983859b5a87475500ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b5e38fc7964402a5cde866ca7bae65"
I20260809 23:03:41.483436 13117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bcd8224f95a47e9ae134c24dac99fe2" candidate_uuid: "641872e41e6f4983859b5a87475500ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3"
I20260809 23:03:41.483564 12970 raft_consensus.cc:3037] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:41.483688 13117 raft_consensus.cc:3037] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:41.485985 13117 raft_consensus.cc:2454] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641872e41e6f4983859b5a87475500ca in term 1.
I20260809 23:03:41.486007 12970 raft_consensus.cc:2454] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 641872e41e6f4983859b5a87475500ca in term 1.
I20260809 23:03:41.486413 12735 leader_election.cc:302] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 641872e41e6f4983859b5a87475500ca, fc22d9fe3f8246febe0d7cad5e6ec9f3; no voters:
I20260809 23:03:41.486754 13194 raft_consensus.cc:2781] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:41.487231 13194 raft_consensus.cc:686] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 1 LEADER]: Becoming Leader. State: Replica: 641872e41e6f4983859b5a87475500ca, State: Running, Role: LEADER
I20260809 23:03:41.487720 13194 consensus_queue.cc:237] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } }
I20260809 23:03:41.492653 12653 catalog_manager.cc:5495] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 641872e41e6f4983859b5a87475500ca (127.24.31.193). New cstate: current_term: 1 leader_uuid: "641872e41e6f4983859b5a87475500ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "641872e41e6f4983859b5a87475500ca" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 } health_report { overall_health: UNKNOWN } } }
I20260809 23:03:41.539502 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.533273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34644 (local address 127.24.31.193:34453)
0809 23:03:41.533511 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.533520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.533544 (+ 24us) server_negotiation.cc:407] Connection header received
0809 23:03:41.533881 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.533885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.533991 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.534167 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:03:41.534620 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.535421 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.537007 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.537493 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.537652 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.538259 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:03:41.538803 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:03:41.538979 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.539098 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.539186 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 23:03:41.548666 12970 raft_consensus.cc:1261] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [term 1 FOLLOWER]: Refusing update from remote peer 641872e41e6f4983859b5a87475500ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:41.548666 13117 raft_consensus.cc:1261] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [term 1 FOLLOWER]: Refusing update from remote peer 641872e41e6f4983859b5a87475500ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:41.549759 13200 consensus_queue.cc:1035] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "13b5e38fc7964402a5cde866ca7bae65" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:41.550174 13194 consensus_queue.cc:1035] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc22d9fe3f8246febe0d7cad5e6ec9f3" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 41427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:41.578178 12653 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:41.578397 12653 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:41.582417 12653 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80e7691817b44f1fa871280beccf95cd: Sending DeleteTablet for 3 replicas of tablet 0bcd8224f95a47e9ae134c24dac99fe2
I20260809 23:03:41.583638 12790 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bcd8224f95a47e9ae134c24dac99fe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:41 UTC) from {username='slave'} at 127.0.0.1:34630
I20260809 23:03:41.583827 12944 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bcd8224f95a47e9ae134c24dac99fe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:41 UTC) from {username='slave'} at 127.0.0.1:59870
I20260809 23:03:41.584409 13092 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bcd8224f95a47e9ae134c24dac99fe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:41 UTC) from {username='slave'} at 127.0.0.1:49566
I20260809 23:03:41.584779 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 12715
I20260809 23:03:41.597806 13216 tablet_replica.cc:330] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: stopping tablet replica
I20260809 23:03:41.598435 13216 raft_consensus.cc:2229] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:41.599118 13216 raft_consensus.cc:2258] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:41.599489 13215 tablet_replica.cc:330] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: stopping tablet replica
I20260809 23:03:41.600383 13215 raft_consensus.cc:2229] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:41.600955 13215 raft_consensus.cc:2258] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:41.601862 13216 ts_tablet_manager.cc:1905] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:41.603449 13215 ts_tablet_manager.cc:1905] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:41.607565 13214 tablet_replica.cc:330] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca: stopping tablet replica
I20260809 23:03:41.608269 13214 raft_consensus.cc:2229] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:03:41.610103 13216 ts_tablet_manager.cc:1918] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:41.610359 13216 log.cc:1196] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/0bcd8224f95a47e9ae134c24dac99fe2
I20260809 23:03:41.611009 13216 ts_tablet_manager.cc:1939] T 0bcd8224f95a47e9ae134c24dac99fe2 P fc22d9fe3f8246febe0d7cad5e6ec9f3: Deleting consensus metadata
I20260809 23:03:41.611315 13214 raft_consensus.cc:2258] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:41.613284 12632 catalog_manager.cc:4841] TS fc22d9fe3f8246febe0d7cad5e6ec9f3 (127.24.31.195:41427): tablet 0bcd8224f95a47e9ae134c24dac99fe2 (table test-table [id=f477d18c7bab4fa28d71be53fc92703e]) successfully deleted
I20260809 23:03:41.614928 13214 ts_tablet_manager.cc:1905] T 0bcd8224f95a47e9ae134c24dac99fe2 P 641872e41e6f4983859b5a87475500ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:41.616496 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.587748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.24.31.193:34453)
0809 23:03:41.587957 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.587963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.607377 (+ 19414us) server_negotiation.cc:407] Connection header received
0809 23:03:41.607746 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.607753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.607880 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.607962 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:41.608739 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.610061 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.612238 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.612940 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.613050 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.614629 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:41.614639 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:41.614656 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:41.615475 (+ 819us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:41.615552 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:41.615583 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:41.615662 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.615943 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.616085 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 23:03:41.617765 13215 ts_tablet_manager.cc:1918] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:41.617847 12863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:41.617992 13215 log.cc:1196] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/0bcd8224f95a47e9ae134c24dac99fe2
I20260809 23:03:41.618881 13215 ts_tablet_manager.cc:1939] T 0bcd8224f95a47e9ae134c24dac99fe2 P 13b5e38fc7964402a5cde866ca7bae65: Deleting consensus metadata
I20260809 23:03:41.621400 12632 catalog_manager.cc:4841] TS 13b5e38fc7964402a5cde866ca7bae65 (127.24.31.194:39627): tablet 0bcd8224f95a47e9ae134c24dac99fe2 (table test-table [id=f477d18c7bab4fa28d71be53fc92703e]) successfully deleted
I20260809 23:03:41.715854 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 12715
W20260809 23:03:41.736301 12631 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.31.193:34453 (error 108)
I20260809 23:03:41.736588 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 12873
I20260809 23:03:41.738420 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.737934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34453 (local address 127.0.0.1:34660)
0809 23:03:41.738093 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.738236 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 23:03:41.738691 12631 catalog_manager.cc:4584] TS 641872e41e6f4983859b5a87475500ca (127.24.31.193:34453): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0bcd8224f95a47e9ae134c24dac99fe2: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34453: connect: Connection refused (error 111)
I20260809 23:03:41.744086 13183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.737604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59880 (local address 127.24.31.194:39627)
0809 23:03:41.737838 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.737845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.737867 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:03:41.737939 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.737945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.738078 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.738186 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:41.738731 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.739566 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.740889 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.741500 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.741634 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.742715 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:41.742723 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:41.742737 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:41.743321 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:41.743366 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:41.743386 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:41.743511 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.743645 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.743750 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 23:03:41.744817 13013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:41.792845 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.792210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34453 (local address 127.0.0.1:34676)
0809 23:03:41.792491 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.792635 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 23:03:41.794147 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.793729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34453 (local address 127.0.0.1:34678)
0809 23:03:41.793896 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.794005 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 23:03:41.837913 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 12873
I20260809 23:03:41.854894 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.854289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34453 (local address 127.0.0.1:34692)
0809 23:03:41.854523 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.854625 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 23:03:41.856264 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.855829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34453 (local address 127.0.0.1:34700)
0809 23:03:41.856016 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.856128 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 23:03:41.859266 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 13022
I20260809 23:03:41.866374 13185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.860295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.24.31.195:41427)
0809 23:03:41.860462 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:41.860468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:41.860488 (+ 20us) server_negotiation.cc:407] Connection header received
0809 23:03:41.860567 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:41.860570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:41.860656 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:41.860747 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:41.861321 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.862132 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.863484 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:41.863925 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:41.864020 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:41.865090 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:41.865098 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:41.865113 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:41.865753 (+ 640us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:41.865809 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:41.865831 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:41.865919 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:41.866058 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 23:03:41.866145 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 23:03:41.867060 13163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:41.937131 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.936627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34453 (local address 127.0.0.1:34714)
0809 23:03:41.936866 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.936976 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 23:03:41.938320 13181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:41.937927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34453 (local address 127.0.0.1:34724)
0809 23:03:41.938124 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:41.938195 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 23:03:41.957489 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 13022
I20260809 23:03:41.976881 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 12613
I20260809 23:03:41.978053 12685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:42.088150 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 12613
2026-08-09T23:03:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2804 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:03:42 dist-test-slave-vxj2 krb5kdc[13228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:03:42 dist-test-slave-vxj2 krb5kdc[13228](info): set up 2 sockets
Aug 09 23:03:42 dist-test-slave-vxj2 krb5kdc[13228](info): commencing operation
krb5kdc: starting...
W20260809 23:03:43.278370 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:03:43 dist-test-slave-vxj2 krb5kdc[13228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:03:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:03:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:03:43.565629 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:46413
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:46413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:43.916694 13244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:43.916981 13244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:43.917073 13244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:43.924160 13244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:03:43.924266 13244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:43.924311 13244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:43.924360 13244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:03:43.924404 13244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:03:43.934737 13244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:46413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:46413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:43.936045 13244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:43.937295 13244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:43.949374 13253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:43.949337 13251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:43.949302 13250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:43.950537 13244 server_base.cc:1034] running on GCE node
I20260809 23:03:43.951412 13244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:43.952880 13244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:43.954090 13244 hybrid_clock.cc:648] HybridClock initialized: now 1786316623954018 us; error 71 us; skew 500 ppm
I20260809 23:03:43.954473 13244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:43 dist-test-slave-vxj2 krb5kdc[13228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316623, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:43.961453 13244 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:03:43.963100 13244 webserver.cc:460] Webserver started at http://127.24.31.254:43363/ using document root <none> and password file <none>
I20260809 23:03:43.963768 13244 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:43.963869 13244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:43.964116 13244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:43.967195 13244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "d65c638a83ed4e12bc0f3daca84f7faa"
format_stamp: "Formatted at 2026-08-09 23:03:43 on dist-test-slave-vxj2"
server_key: "1c56af07c77e2eb83d75a2d18da6069d"
server_key_iv: "f3c2d83ad09a974c40f69d4495d37d9e"
server_key_version: "encryptionkey@0"
I20260809 23:03:43.967890 13244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "d65c638a83ed4e12bc0f3daca84f7faa"
format_stamp: "Formatted at 2026-08-09 23:03:43 on dist-test-slave-vxj2"
server_key: "1c56af07c77e2eb83d75a2d18da6069d"
server_key_iv: "f3c2d83ad09a974c40f69d4495d37d9e"
server_key_version: "encryptionkey@0"
I20260809 23:03:43.972981 13244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 23:03:43.976390 13260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:43.978024 13244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:03:43.978222 13244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "d65c638a83ed4e12bc0f3daca84f7faa"
format_stamp: "Formatted at 2026-08-09 23:03:43 on dist-test-slave-vxj2"
server_key: "1c56af07c77e2eb83d75a2d18da6069d"
server_key_iv: "f3c2d83ad09a974c40f69d4495d37d9e"
server_key_version: "encryptionkey@0"
I20260809 23:03:43.978440 13244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:43.989585 13244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:43.993866 13244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:43.994165 13244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:44.013710 13244 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:46413
I20260809 23:03:44.013738 13324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:46413 every 8 connection(s)
I20260809 23:03:44.015156 13244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:03:44.017830 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 13244
I20260809 23:03:44.018033 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:03:44.018477 24703 external_mini_cluster.cc:1428] Setting key 367c852ded540492175f88fba78c2cb7
I20260809 23:03:44.019603 13325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 23:03:44 dist-test-slave-vxj2 krb5kdc[13228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:44.034399 13325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa: Bootstrap starting.
I20260809 23:03:44.038511 13325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:44.039865 13325 log.cc:824] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:44.042847 13325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa: No bootstrap required, opened a new log
I20260809 23:03:44.044464 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:44.020416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.24.31.254:46413)
0809 23:03:44.021361 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:44.021377 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:44.021433 (+ 56us) server_negotiation.cc:407] Connection header received
0809 23:03:44.023417 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:44.023445 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:44.023973 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:44.024496 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:44.026154 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:44.028380 (+ 2226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:44.029863 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:44.030910 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:44.031107 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:44.035464 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:44.035475 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:44.035493 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:44.035535 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:44.039828 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:44.040730 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:44.040738 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:44.040743 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:44.040940 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:44.041493 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:44.041498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:44.041501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:44.042075 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:44.042424 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:44.042957 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 23:03:44.043422 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":377,"threads_started":1}
I20260809 23:03:44.051535 13325 raft_consensus.cc:348] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c638a83ed4e12bc0f3daca84f7faa" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46413 } }
I20260809 23:03:44.051927 13325 raft_consensus.cc:374] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:44.052037 13325 raft_consensus.cc:729] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d65c638a83ed4e12bc0f3daca84f7faa, State: Initialized, Role: FOLLOWER
I20260809 23:03:44.052577 13325 consensus_queue.cc:260] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c638a83ed4e12bc0f3daca84f7faa" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46413 } }
I20260809 23:03:44.052814 13325 raft_consensus.cc:388] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:03:44.052961 13325 raft_consensus.cc:482] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:03:44.053138 13325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:44.055375 13325 raft_consensus.cc:504] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c638a83ed4e12bc0f3daca84f7faa" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46413 } }
I20260809 23:03:44.055855 13325 leader_election.cc:302] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d65c638a83ed4e12bc0f3daca84f7faa; no voters:
I20260809 23:03:44.056558 13325 leader_election.cc:288] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:03:44.056687 13330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:44.057651 13330 raft_consensus.cc:686] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [term 1 LEADER]: Becoming Leader. State: Replica: d65c638a83ed4e12bc0f3daca84f7faa, State: Running, Role: LEADER
I20260809 23:03:44.058311 13330 consensus_queue.cc:237] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c638a83ed4e12bc0f3daca84f7faa" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46413 } }
I20260809 23:03:44.058960 13325 sys_catalog.cc:564] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:03:44.061389 13332 sys_catalog.cc:455] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d65c638a83ed4e12bc0f3daca84f7faa. Latest consensus state: current_term: 1 leader_uuid: "d65c638a83ed4e12bc0f3daca84f7faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c638a83ed4e12bc0f3daca84f7faa" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46413 } } }
I20260809 23:03:44.061918 13332 sys_catalog.cc:458] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:44.061462 13331 sys_catalog.cc:455] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d65c638a83ed4e12bc0f3daca84f7faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d65c638a83ed4e12bc0f3daca84f7faa" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 46413 } } }
I20260809 23:03:44.062875 13331 sys_catalog.cc:458] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:44.064337 13339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:03:44.068888 13339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:03:44.077636 13339 catalog_manager.cc:1347] Generated new cluster ID: e84545a5c53a461e8ef2268d26ea493b
I20260809 23:03:44.077787 13339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:03:44.102982 13339 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:03:44.104458 13339 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:03:44.115808 13339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa: Generated new TSK 0
I20260809 23:03:44.116851 13339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 23:03:44.192188 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46413
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:44.533078 13355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:44.533370 13355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:44.533463 13355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:44.540680 13355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:44.540827 13355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:44.540946 13355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:03:44.551218 13355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:44.552520 13355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:44.553916 13355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:44.567572 13364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:44.567646 13367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:44.567641 13362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:44.568490 13355 server_base.cc:1034] running on GCE node
I20260809 23:03:44.569205 13355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:44.570168 13355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:44.571421 13355 hybrid_clock.cc:648] HybridClock initialized: now 1786316624571369 us; error 39 us; skew 500 ppm
I20260809 23:03:44.571828 13355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:44 dist-test-slave-vxj2 krb5kdc[13228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316624, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:44.575280 13355 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:03:44.576844 13355 webserver.cc:460] Webserver started at http://127.24.31.193:40561/ using document root <none> and password file <none>
I20260809 23:03:44.577602 13355 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:44.577755 13355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:44.578128 13355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:44.581068 13355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "4557dd9435de401d93e0689b9af904e9"
format_stamp: "Formatted at 2026-08-09 23:03:44 on dist-test-slave-vxj2"
server_key: "b93ce9fc39ccf809e8807727826db8ab"
server_key_iv: "115126e3f7cbcd64d38279d7aa0bd698"
server_key_version: "encryptionkey@0"
I20260809 23:03:44.581955 13355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "4557dd9435de401d93e0689b9af904e9"
format_stamp: "Formatted at 2026-08-09 23:03:44 on dist-test-slave-vxj2"
server_key: "b93ce9fc39ccf809e8807727826db8ab"
server_key_iv: "115126e3f7cbcd64d38279d7aa0bd698"
server_key_version: "encryptionkey@0"
I20260809 23:03:44.587275 13355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 23:03:44.590747 13376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:44.592120 13355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:03:44.592305 13355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "4557dd9435de401d93e0689b9af904e9"
format_stamp: "Formatted at 2026-08-09 23:03:44 on dist-test-slave-vxj2"
server_key: "b93ce9fc39ccf809e8807727826db8ab"
server_key_iv: "115126e3f7cbcd64d38279d7aa0bd698"
server_key_version: "encryptionkey@0"
I20260809 23:03:44.592515 13355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:44.626673 13355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:44.631023 13355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:44.631330 13355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:44.632299 13355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:44.633795 13355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:44.633899 13355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:44.634064 13355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:44.634140 13355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:44.668658 13355 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:33793
I20260809 23:03:44.668748 13509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:33793 every 8 connection(s)
I20260809 23:03:44.670001 13355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:03:44.679888 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 13355
I20260809 23:03:44.680099 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:03:44.680568 24703 external_mini_cluster.cc:1428] Setting key 9316c3d613e6d223c2aa5d0da8479281
Aug 09 23:03:44 dist-test-slave-vxj2 krb5kdc[13228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316624, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:44.690366 13512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:44.672560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:54787 (local address 127.24.31.254:46413)
0809 23:03:44.673059 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:44.673064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:44.674052 (+ 988us) server_negotiation.cc:407] Connection header received
0809 23:03:44.675902 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:44.675908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:44.676003 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:44.676119 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:44.677602 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:44.678379 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:44.680015 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:44.680466 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:44.680606 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:44.684591 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:44.684596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:44.684599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:44.684630 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:44.687274 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:44.688000 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:44.688005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:44.688013 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:44.688126 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:44.688569 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:44.688573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:44.688576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:44.688865 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:44.689035 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:44.689778 (+ 743us) server_negotiation.cc:299] Negotiation successful
0809 23:03:44.690011 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":252,"threads_started":1}
I20260809 23:03:44.691277 13513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:44.673186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46413 (local address 127.24.31.193:54787)
0809 23:03:44.673896 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:44.673935 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:44.675639 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:44.676356 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:44.676363 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:44.676863 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:44.677440 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:44.677457 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:44.678555 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:44.678559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:44.679862 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:44.679874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:44.680703 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:44.680712 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:44.680961 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:44.681815 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:03:44.681871 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:44.684374 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:44.687417 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:44.687426 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:44.687440 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:44.687871 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:44.688235 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:44.688240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:44.688243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:44.688442 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:44.688996 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:44.689004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:44.689411 (+ 407us) client_negotiation.cc:769] Sending connection context
0809 23:03:44.689668 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 23:03:44.690162 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":253,"threads_started":1}
I20260809 23:03:44.692931 13510 heartbeater.cc:347] Connected to a master server at 127.24.31.254:46413
I20260809 23:03:44.693290 13510 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:44.694229 13510 heartbeater.cc:511] Master 127.24.31.254:46413 requested a full tablet report, sending...
I20260809 23:03:44.696797 13282 ts_manager.cc:194] Registered new tserver with Master: 4557dd9435de401d93e0689b9af904e9 (127.24.31.193:33793)
I20260809 23:03:44.698383 13282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:54787
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 23:03:44.743778 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46413
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:45.075662 13516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:45.075999 13516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:45.076170 13516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:45.083616 13516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:45.083765 13516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:45.083956 13516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:03:45.093940 13516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:45.095288 13516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:45.096714 13516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:45.110004 13527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:45.110038 13522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:45.109972 13523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:45.110920 13516 server_base.cc:1034] running on GCE node
I20260809 23:03:45.111521 13516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:45.112424 13516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:45.113648 13516 hybrid_clock.cc:648] HybridClock initialized: now 1786316625113603 us; error 57 us; skew 500 ppm
I20260809 23:03:45.113961 13516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:45 dist-test-slave-vxj2 krb5kdc[13228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316625, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:45.117012 13516 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:03:45.118489 13516 webserver.cc:460] Webserver started at http://127.24.31.194:33941/ using document root <none> and password file <none>
I20260809 23:03:45.119221 13516 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:45.119396 13516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:45.119707 13516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:45.122646 13516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "5ea10d8d50274dcfad62680668d14d75"
format_stamp: "Formatted at 2026-08-09 23:03:45 on dist-test-slave-vxj2"
server_key: "8cf49e5de2b3c9bef08838e5700fc016"
server_key_iv: "440d75f9b4aafdd01c0603f066dc149e"
server_key_version: "encryptionkey@0"
I20260809 23:03:45.123448 13516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "5ea10d8d50274dcfad62680668d14d75"
format_stamp: "Formatted at 2026-08-09 23:03:45 on dist-test-slave-vxj2"
server_key: "8cf49e5de2b3c9bef08838e5700fc016"
server_key_iv: "440d75f9b4aafdd01c0603f066dc149e"
server_key_version: "encryptionkey@0"
I20260809 23:03:45.128523 13516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 23:03:45.132076 13534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:45.133402 13516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 23:03:45.133574 13516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "5ea10d8d50274dcfad62680668d14d75"
format_stamp: "Formatted at 2026-08-09 23:03:45 on dist-test-slave-vxj2"
server_key: "8cf49e5de2b3c9bef08838e5700fc016"
server_key_iv: "440d75f9b4aafdd01c0603f066dc149e"
server_key_version: "encryptionkey@0"
I20260809 23:03:45.133780 13516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:45.166662 13516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:45.171101 13516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:45.171415 13516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:45.172465 13516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:45.173846 13516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:45.173965 13516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:45.174113 13516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:45.174182 13516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:45.208734 13516 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:44693
I20260809 23:03:45.208829 13666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:44693 every 8 connection(s)
I20260809 23:03:45.210106 13516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:03:45.211158 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 13516
I20260809 23:03:45.211383 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:03:45.212049 24703 external_mini_cluster.cc:1428] Setting key a6deb477c899e394daa212cf5a25ea3c
Aug 09 23:03:45 dist-test-slave-vxj2 krb5kdc[13228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316625, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:45.232491 13671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.213174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:44071 (local address 127.24.31.254:46413)
0809 23:03:45.213628 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:45.213632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:45.214825 (+ 1193us) server_negotiation.cc:407] Connection header received
0809 23:03:45.216692 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:45.216701 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:45.216817 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:45.216946 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:45.218505 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.219667 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.221232 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.222043 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.222204 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.225877 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:45.225883 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:45.225886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:45.225921 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:45.229031 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:45.230039 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:45.230043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:45.230045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:45.230171 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:45.230665 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:45.230668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:45.230670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:45.230968 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:45.231087 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:45.232008 (+ 921us) server_negotiation.cc:299] Negotiation successful
0809 23:03:45.232227 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":210,"threads_started":1}
I20260809 23:03:45.234022 13672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.213798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46413 (local address 127.24.31.194:44071)
0809 23:03:45.214609 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:45.214656 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:45.216415 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:45.217148 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:45.217217 (+ 69us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:45.217748 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:45.218321 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.218339 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.219838 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.219841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.221056 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.221068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.222259 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.222263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.222451 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.223157 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:03:45.223184 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:45.225561 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:45.229205 (+ 3644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:45.229216 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:45.229235 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:45.229894 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:45.230280 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:45.230287 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:45.230293 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:45.230552 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:45.231093 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:45.231104 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:45.231575 (+ 471us) client_negotiation.cc:769] Sending connection context
0809 23:03:45.231889 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 23:03:45.232493 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":374,"threads_started":1}
I20260809 23:03:45.235517 13667 heartbeater.cc:347] Connected to a master server at 127.24.31.254:46413
I20260809 23:03:45.235831 13667 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:45.236572 13667 heartbeater.cc:511] Master 127.24.31.254:46413 requested a full tablet report, sending...
I20260809 23:03:45.238413 13282 ts_manager.cc:194] Registered new tserver with Master: 5ea10d8d50274dcfad62680668d14d75 (127.24.31.194:44693)
I20260809 23:03:45.239346 13282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:44071
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 23:03:45.280727 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46413
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:33143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:45.625890 13675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:45.626225 13675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:45.626402 13675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:45.633643 13675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:45.633839 13675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:45.634023 13675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:03:45.644028 13675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:33143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:46413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:45.645438 13675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:45.646770 13675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:45.660531 13684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:45.660589 13682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:45.661110 13687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:45.662768 13675 server_base.cc:1034] running on GCE node
I20260809 23:03:45.663415 13675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:45.664446 13675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:45.665678 13675 hybrid_clock.cc:648] HybridClock initialized: now 1786316625665606 us; error 54 us; skew 500 ppm
I20260809 23:03:45.666010 13675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:45 dist-test-slave-vxj2 krb5kdc[13228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316625, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:45.669397 13675 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:03:45.671015 13675 webserver.cc:460] Webserver started at http://127.24.31.195:32839/ using document root <none> and password file <none>
I20260809 23:03:45.671702 13675 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:45.671834 13675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:45.672266 13675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:45.675148 13675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "e875bb695cee4a13a73db6e7194d61ad"
format_stamp: "Formatted at 2026-08-09 23:03:45 on dist-test-slave-vxj2"
server_key: "937fc66f4abaf498cfadbe67efecfbb6"
server_key_iv: "ba518dc2bdddebb64da163c13ebb5ef1"
server_key_version: "encryptionkey@0"
I20260809 23:03:45.675907 13675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "e875bb695cee4a13a73db6e7194d61ad"
format_stamp: "Formatted at 2026-08-09 23:03:45 on dist-test-slave-vxj2"
server_key: "937fc66f4abaf498cfadbe67efecfbb6"
server_key_iv: "ba518dc2bdddebb64da163c13ebb5ef1"
server_key_version: "encryptionkey@0"
I20260809 23:03:45.681104 13675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 23:03:45.684679 13694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:45.686115 13675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:03:45.686298 13675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "e875bb695cee4a13a73db6e7194d61ad"
format_stamp: "Formatted at 2026-08-09 23:03:45 on dist-test-slave-vxj2"
server_key: "937fc66f4abaf498cfadbe67efecfbb6"
server_key_iv: "ba518dc2bdddebb64da163c13ebb5ef1"
server_key_version: "encryptionkey@0"
I20260809 23:03:45.686527 13675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:45.702020 13510 heartbeater.cc:503] Master 127.24.31.254:46413 was elected leader, sending a full tablet report...
I20260809 23:03:45.731438 13675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:45.736056 13675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:45.736380 13675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:45.737507 13675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:45.739042 13675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:45.739146 13675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:45.739305 13675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:45.739379 13675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:45.775081 13675 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:46629
I20260809 23:03:45.775130 13822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:46629 every 8 connection(s)
I20260809 23:03:45.776434 13675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:03:45.779511 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 13675
I20260809 23:03:45.779706 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:03:45.780381 24703 external_mini_cluster.cc:1428] Setting key b955ec456090deb2e587944dc5c6d19c
Aug 09 23:03:45 dist-test-slave-vxj2 krb5kdc[13228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316625, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:45.797993 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.779832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:52877 (local address 127.24.31.254:46413)
0809 23:03:45.780354 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:45.780363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:45.780970 (+ 607us) server_negotiation.cc:407] Connection header received
0809 23:03:45.783345 (+ 2375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:45.783352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:45.783456 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:45.783566 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:45.785345 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.786136 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.787679 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.788210 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.788308 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.791921 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:45.791926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:45.791929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:45.791962 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:45.794459 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:45.795434 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:45.795438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:45.795440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:45.795599 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:45.796150 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:45.796157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:45.796162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:45.796540 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:45.796684 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:45.797550 (+ 866us) server_negotiation.cc:299] Negotiation successful
0809 23:03:45.797710 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":247,"threads_started":1}
I20260809 23:03:45.798931 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.779926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:46413 (local address 127.24.31.195:52877)
0809 23:03:45.780754 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:45.780797 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:45.783017 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:45.783971 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:45.783979 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:45.784549 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:45.785124 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.785141 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.786275 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.786279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.787543 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.787556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.788407 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.788412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.788621 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.789333 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:03:45.789356 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:45.791681 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:45.794634 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:45.794640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:45.794652 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:45.795280 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:45.795705 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:45.795709 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:45.795711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:45.796020 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:45.796727 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:45.796759 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:45.797093 (+ 334us) client_negotiation.cc:769] Sending connection context
0809 23:03:45.797412 (+ 319us) client_negotiation.cc:240] Negotiation successful
0809 23:03:45.797826 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":345,"threads_started":1}
I20260809 23:03:45.800200 13823 heartbeater.cc:347] Connected to a master server at 127.24.31.254:46413
I20260809 23:03:45.800511 13823 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:45.801287 13823 heartbeater.cc:511] Master 127.24.31.254:46413 requested a full tablet report, sending...
I20260809 23:03:45.802619 13282 ts_manager.cc:194] Registered new tserver with Master: e875bb695cee4a13a73db6e7194d61ad (127.24.31.195:46629)
I20260809 23:03:45.803766 13282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:52877
I20260809 23:03:45.808410 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:03:45.826215 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.812632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.24.31.254:46413)
0809 23:03:45.812835 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:45.812844 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:45.813366 (+ 522us) server_negotiation.cc:407] Connection header received
0809 23:03:45.813505 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:45.813509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:45.813569 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:45.813651 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:45.815069 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.815800 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.817034 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.817497 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.817594 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.820572 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:45.820577 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:45.820580 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:45.820612 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:45.823312 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:45.824187 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:45.824191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:45.824194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:45.824313 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:45.824802 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:45.824806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:45.824808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:45.825104 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:45.825410 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:45.825741 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 23:03:45.825884 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 23:03:45.830652 13282 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:03:45.837774 13282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:03:45.876516 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.851134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39338 (local address 127.24.31.195:46629)
0809 23:03:45.866589 (+ 15455us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:45.866602 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:45.867539 (+ 937us) server_negotiation.cc:407] Connection header received
0809 23:03:45.867657 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:45.867665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:45.867958 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:45.868157 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:45.869401 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.871073 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.874069 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.875425 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.875554 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.875807 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:45.875945 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 23:03:45.876136 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15202,"thread_start_us":15000,"threads_started":1}
I20260809 23:03:45.876575 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.850407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:46629 (local address 127.0.0.1:39338)
0809 23:03:45.867151 (+ 16744us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:45.867176 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:45.867373 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:45.868507 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:45.868515 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:45.868551 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:45.869193 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.869210 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.871216 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.871222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.873913 (+ 2691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.873928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.875606 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.875610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.875740 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.875775 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 23:03:45.875882 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 23:03:45.876077 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16567,"mutex_wait_us":159,"spinlock_wait_cycles":896,"thread_start_us":16848,"threads_started":1}
I20260809 23:03:45.877897 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.850326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56300 (local address 127.24.31.194:44693)
0809 23:03:45.867869 (+ 17543us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:45.867877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:45.867903 (+ 26us) server_negotiation.cc:407] Connection header received
0809 23:03:45.867994 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:45.868000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:45.868188 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:45.868393 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:45.869770 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.871272 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.873844 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.875408 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.875555 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.875757 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:45.877387 (+ 1630us) server_negotiation.cc:299] Negotiation successful
0809 23:03:45.877556 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17386,"thread_start_us":17246,"threads_started":1}
I20260809 23:03:45.878621 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.851131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36828 (local address 127.24.31.193:33793)
0809 23:03:45.863341 (+ 12210us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:45.863355 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:45.867855 (+ 4500us) server_negotiation.cc:407] Connection header received
0809 23:03:45.867970 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:45.867978 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:45.868206 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:45.868389 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:45.869745 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.870774 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.875871 (+ 5097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.877605 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.877746 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.878050 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:45.878286 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 23:03:45.878406 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11960,"thread_start_us":11759,"threads_started":1}
I20260809 23:03:45.878618 13842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.850596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:33793 (local address 127.0.0.1:36828)
0809 23:03:45.867548 (+ 16952us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:45.867569 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:45.867698 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:45.868639 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:45.868644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:45.868695 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:45.869546 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.869557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.870923 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.870931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.873920 (+ 2989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.873928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.877843 (+ 3915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.877849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.878040 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.878073 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 23:03:45.878154 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 23:03:45.878272 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16819,"mutex_wait_us":139,"spinlock_wait_cycles":4224,"thread_start_us":16219,"threads_started":1}
I20260809 23:03:45.877642 13839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:45.850114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:44693 (local address 127.0.0.1:56300)
0809 23:03:45.864013 (+ 13899us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:45.864063 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:45.864331 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:45.868639 (+ 4308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:45.868644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:45.868695 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:45.869519 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.869539 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.871411 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.871419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.873673 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:45.873686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:45.876974 (+ 3288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:45.876980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:45.877150 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:45.877212 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 23:03:45.877287 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 23:03:45.877374 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13566,"mutex_wait_us":27,"spinlock_wait_cycles":19584,"thread_start_us":13404,"threads_started":1}
I20260809 23:03:45.881458 13748 tablet_service.cc:1467] Processing CreateTablet for tablet cb2b2309a4df4099a0562c42ddf86da4 (DEFAULT_TABLE table=test-table [id=ff8a116cf8c341cab640d0f81b6b63f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:45.881798 13435 tablet_service.cc:1467] Processing CreateTablet for tablet cb2b2309a4df4099a0562c42ddf86da4 (DEFAULT_TABLE table=test-table [id=ff8a116cf8c341cab640d0f81b6b63f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:45.883031 13435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb2b2309a4df4099a0562c42ddf86da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:45.883101 13748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb2b2309a4df4099a0562c42ddf86da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:45.884763 13588 tablet_service.cc:1467] Processing CreateTablet for tablet cb2b2309a4df4099a0562c42ddf86da4 (DEFAULT_TABLE table=test-table [id=ff8a116cf8c341cab640d0f81b6b63f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:45.886111 13588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb2b2309a4df4099a0562c42ddf86da4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:45.896204 13847 tablet_bootstrap.cc:492] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: Bootstrap starting.
I20260809 23:03:45.897809 13848 tablet_bootstrap.cc:492] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: Bootstrap starting.
I20260809 23:03:45.899921 13847 tablet_bootstrap.cc:654] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:45.900152 13849 tablet_bootstrap.cc:492] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: Bootstrap starting.
I20260809 23:03:45.901242 13847 log.cc:824] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:45.901479 13848 tablet_bootstrap.cc:654] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:45.902544 13848 log.cc:824] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:45.903391 13849 tablet_bootstrap.cc:654] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:45.904559 13847 tablet_bootstrap.cc:492] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: No bootstrap required, opened a new log
I20260809 23:03:45.904629 13849 log.cc:824] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:45.904875 13847 ts_tablet_manager.cc:1397] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 23:03:45.905994 13848 tablet_bootstrap.cc:492] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: No bootstrap required, opened a new log
I20260809 23:03:45.906286 13848 ts_tablet_manager.cc:1397] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260809 23:03:45.908016 13849 tablet_bootstrap.cc:492] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: No bootstrap required, opened a new log
I20260809 23:03:45.908269 13849 ts_tablet_manager.cc:1397] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 23:03:45.912820 13847 raft_consensus.cc:348] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } }
I20260809 23:03:45.913193 13847 raft_consensus.cc:374] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:45.913298 13847 raft_consensus.cc:729] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4557dd9435de401d93e0689b9af904e9, State: Initialized, Role: FOLLOWER
I20260809 23:03:45.913861 13847 consensus_queue.cc:260] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } }
I20260809 23:03:45.917003 13848 raft_consensus.cc:348] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } }
I20260809 23:03:45.917470 13848 raft_consensus.cc:374] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:45.917589 13848 raft_consensus.cc:729] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e875bb695cee4a13a73db6e7194d61ad, State: Initialized, Role: FOLLOWER
I20260809 23:03:45.918260 13848 consensus_queue.cc:260] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } }
I20260809 23:03:45.918795 13847 ts_tablet_manager.cc:1428] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: Time spent starting tablet: real 0.014s user 0.004s sys 0.011s
I20260809 23:03:45.918917 13849 raft_consensus.cc:348] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } }
I20260809 23:03:45.919425 13849 raft_consensus.cc:374] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:45.919540 13849 raft_consensus.cc:729] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ea10d8d50274dcfad62680668d14d75, State: Initialized, Role: FOLLOWER
I20260809 23:03:45.920012 13823 heartbeater.cc:503] Master 127.24.31.254:46413 was elected leader, sending a full tablet report...
I20260809 23:03:45.920509 13848 ts_tablet_manager.cc:1428] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260809 23:03:45.920971 13849 consensus_queue.cc:260] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } }
I20260809 23:03:45.922214 13667 heartbeater.cc:503] Master 127.24.31.254:46413 was elected leader, sending a full tablet report...
I20260809 23:03:45.922856 13849 ts_tablet_manager.cc:1428] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
W20260809 23:03:45.922886 13511 tablet.cc:2369] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:03:45.962236 13670 tablet.cc:2369] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:46.007231 13856 raft_consensus.cc:482] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:03:46.007580 13856 raft_consensus.cc:504] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } }
I20260809 23:03:46.009440 13856 leader_election.cc:288] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e875bb695cee4a13a73db6e7194d61ad (127.24.31.195:46629), 4557dd9435de401d93e0689b9af904e9 (127.24.31.193:33793)
I20260809 23:03:46.016731 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:46.009985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:56199 (local address 127.24.31.195:46629)
0809 23:03:46.010255 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:46.010263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:46.011227 (+ 964us) server_negotiation.cc:407] Connection header received
0809 23:03:46.011480 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:46.011484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:46.011547 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:46.011645 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:46.012271 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.013003 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.014986 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.015781 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.015979 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:46.016068 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:46.016345 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 23:03:46.016476 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260809 23:03:46.016799 13857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:46.009651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:46629 (local address 127.24.31.194:56199)
0809 23:03:46.011096 (+ 1445us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:46.011112 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:46.011313 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:46.011664 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:46.011668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:46.011694 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:46.012097 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:46.012108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.013136 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.013144 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:46.014813 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:46.014826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.015998 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.016004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:46.016163 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:46.016197 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 23:03:46.016292 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 23:03:46.016418 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"mutex_wait_us":1,"thread_start_us":1017,"threads_started":1}
I20260809 23:03:46.017920 13774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb2b2309a4df4099a0562c42ddf86da4" candidate_uuid: "5ea10d8d50274dcfad62680668d14d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e875bb695cee4a13a73db6e7194d61ad" is_pre_election: true
I20260809 23:03:46.018205 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:46.010018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:39949 (local address 127.24.31.193:33793)
0809 23:03:46.010506 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:46.010516 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:46.011459 (+ 943us) server_negotiation.cc:407] Connection header received
0809 23:03:46.011661 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:46.011666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:46.011735 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:46.011836 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:46.012428 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.013302 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.015880 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.017048 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.017238 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:46.017304 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:46.017820 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 23:03:46.017959 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257}
I20260809 23:03:46.018517 13774 raft_consensus.cc:2454] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ea10d8d50274dcfad62680668d14d75 in term 0.
I20260809 23:03:46.018981 13858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:46.009985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:33793 (local address 127.24.31.194:39949)
0809 23:03:46.011347 (+ 1362us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:46.011362 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:46.011518 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:46.011855 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:46.011859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:46.011889 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:46.012281 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:46.012290 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.013494 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.013499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:46.015708 (+ 2209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:46.015723 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.017443 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.017449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:46.017615 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:46.017643 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 23:03:46.018445 (+ 802us) client_negotiation.cc:240] Negotiation successful
0809 23:03:46.018585 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1237,"mutex_wait_us":28,"thread_start_us":823,"threads_started":1}
I20260809 23:03:46.019223 13538 leader_election.cc:302] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ea10d8d50274dcfad62680668d14d75, e875bb695cee4a13a73db6e7194d61ad; no voters:
I20260809 23:03:46.019600 13856 raft_consensus.cc:2781] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:03:46.019722 13856 raft_consensus.cc:482] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:03:46.019814 13856 raft_consensus.cc:3037] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:46.019958 13460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb2b2309a4df4099a0562c42ddf86da4" candidate_uuid: "5ea10d8d50274dcfad62680668d14d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4557dd9435de401d93e0689b9af904e9" is_pre_election: true
I20260809 23:03:46.020515 13460 raft_consensus.cc:2454] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ea10d8d50274dcfad62680668d14d75 in term 0.
I20260809 23:03:46.023137 13856 raft_consensus.cc:504] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } }
I20260809 23:03:46.023818 13856 leader_election.cc:288] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [CANDIDATE]: Term 1 election: Requested vote from peers e875bb695cee4a13a73db6e7194d61ad (127.24.31.195:46629), 4557dd9435de401d93e0689b9af904e9 (127.24.31.193:33793)
I20260809 23:03:46.024336 13460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb2b2309a4df4099a0562c42ddf86da4" candidate_uuid: "5ea10d8d50274dcfad62680668d14d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4557dd9435de401d93e0689b9af904e9"
I20260809 23:03:46.024353 13774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb2b2309a4df4099a0562c42ddf86da4" candidate_uuid: "5ea10d8d50274dcfad62680668d14d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e875bb695cee4a13a73db6e7194d61ad"
I20260809 23:03:46.024544 13774 raft_consensus.cc:3037] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:46.024544 13460 raft_consensus.cc:3037] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:46.026912 13460 raft_consensus.cc:2454] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ea10d8d50274dcfad62680668d14d75 in term 1.
I20260809 23:03:46.027384 13537 leader_election.cc:302] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4557dd9435de401d93e0689b9af904e9, 5ea10d8d50274dcfad62680668d14d75; no voters:
I20260809 23:03:46.027688 13856 raft_consensus.cc:2781] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:46.027734 13774 raft_consensus.cc:2454] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ea10d8d50274dcfad62680668d14d75 in term 1.
W20260809 23:03:46.028194 13824 tablet.cc:2369] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:46.031188 13856 raft_consensus.cc:686] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 1 LEADER]: Becoming Leader. State: Replica: 5ea10d8d50274dcfad62680668d14d75, State: Running, Role: LEADER
I20260809 23:03:46.031695 13856 consensus_queue.cc:237] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } }
I20260809 23:03:46.036546 13282 catalog_manager.cc:5495] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ea10d8d50274dcfad62680668d14d75 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "5ea10d8d50274dcfad62680668d14d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ea10d8d50274dcfad62680668d14d75" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 44693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 } health_report { overall_health: UNKNOWN } } }
I20260809 23:03:46.069571 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:46.063559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56310 (local address 127.24.31.194:44693)
0809 23:03:46.063849 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:46.063856 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:46.064081 (+ 225us) server_negotiation.cc:407] Connection header received
0809 23:03:46.064359 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:46.064363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:46.064449 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:46.064609 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:03:46.065040 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.065795 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.067207 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.067629 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.067806 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:46.068401 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:03:46.068918 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:03:46.069105 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:46.069275 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 23:03:46.069321 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 23:03:46.077769 13460 raft_consensus.cc:1261] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [term 1 FOLLOWER]: Refusing update from remote peer 5ea10d8d50274dcfad62680668d14d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:46.077811 13774 raft_consensus.cc:1261] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [term 1 FOLLOWER]: Refusing update from remote peer 5ea10d8d50274dcfad62680668d14d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:46.078754 13859 consensus_queue.cc:1035] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4557dd9435de401d93e0689b9af904e9" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 33793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:46.079108 13856 consensus_queue.cc:1035] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e875bb695cee4a13a73db6e7194d61ad" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:46.091697 13864 mvcc.cc:205] Tried to move back new op lower bound from 7316752900404334592 to 7316752900230004736. Current Snapshot: MvccSnapshot[applied={T|T < 7316752900404334592}]
I20260809 23:03:46.093906 13865 mvcc.cc:205] Tried to move back new op lower bound from 7316752900404334592 to 7316752900230004736. Current Snapshot: MvccSnapshot[applied={T|T < 7316752900404334592}]
I20260809 23:03:46.094185 13863 mvcc.cc:205] Tried to move back new op lower bound from 7316752900404334592 to 7316752900230004736. Current Snapshot: MvccSnapshot[applied={T|T < 7316752900404334592 or (T in {7316752900404334592})}]
I20260809 23:03:46.104559 13280 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:46.104877 13280 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:46.108752 13280 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d65c638a83ed4e12bc0f3daca84f7faa: Sending DeleteTablet for 3 replicas of tablet cb2b2309a4df4099a0562c42ddf86da4
I20260809 23:03:46.109997 13748 tablet_service.cc:1514] Processing DeleteTablet for tablet cb2b2309a4df4099a0562c42ddf86da4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:46 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:39338
I20260809 23:03:46.109994 13588 tablet_service.cc:1514] Processing DeleteTablet for tablet cb2b2309a4df4099a0562c42ddf86da4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:46 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:56300
I20260809 23:03:46.110594 13435 tablet_service.cc:1514] Processing DeleteTablet for tablet cb2b2309a4df4099a0562c42ddf86da4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:46 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:36828
I20260809 23:03:46.114696 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 13355
I20260809 23:03:46.122071 13874 tablet_replica.cc:330] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: stopping tablet replica
I20260809 23:03:46.122527 13874 raft_consensus.cc:2229] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:46.123056 13874 raft_consensus.cc:2258] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:46.127496 13873 tablet_replica.cc:330] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: stopping tablet replica
I20260809 23:03:46.127914 13873 raft_consensus.cc:2229] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:46.128401 13873 raft_consensus.cc:2258] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:46.129616 13874 ts_tablet_manager.cc:1905] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:46.130328 13873 ts_tablet_manager.cc:1905] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 23:03:46 dist-test-slave-vxj2 krb5kdc[13228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 23:03:46.137379 13872 tablet_replica.cc:330] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: stopping tablet replica
I20260809 23:03:46.137671 13873 ts_tablet_manager.cc:1918] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:46.137832 13873 log.cc:1196] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/cb2b2309a4df4099a0562c42ddf86da4
I20260809 23:03:46.137827 13872 raft_consensus.cc:2229] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:03:46.138392 13873 ts_tablet_manager.cc:1939] T cb2b2309a4df4099a0562c42ddf86da4 P e875bb695cee4a13a73db6e7194d61ad: Deleting consensus metadata
I20260809 23:03:46.138477 13872 raft_consensus.cc:2258] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:46.140007 13874 ts_tablet_manager.cc:1918] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:46.140236 13874 log.cc:1196] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/cb2b2309a4df4099a0562c42ddf86da4
I20260809 23:03:46.140697 13264 catalog_manager.cc:4841] TS e875bb695cee4a13a73db6e7194d61ad (127.24.31.195:46629): tablet cb2b2309a4df4099a0562c42ddf86da4 (table test-table [id=ff8a116cf8c341cab640d0f81b6b63f2]) successfully deleted
I20260809 23:03:46.140924 13874 ts_tablet_manager.cc:1939] T cb2b2309a4df4099a0562c42ddf86da4 P 4557dd9435de401d93e0689b9af904e9: Deleting consensus metadata
I20260809 23:03:46.142113 13872 ts_tablet_manager.cc:1905] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:46.142953 13263 catalog_manager.cc:4841] TS 4557dd9435de401d93e0689b9af904e9 (127.24.31.193:33793): tablet cb2b2309a4df4099a0562c42ddf86da4 (table test-table [id=ff8a116cf8c341cab640d0f81b6b63f2]) successfully deleted
I20260809 23:03:46.145879 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:46.116278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.24.31.193:33793)
0809 23:03:46.116507 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:46.116517 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:46.121296 (+ 4779us) server_negotiation.cc:407] Connection header received
0809 23:03:46.123728 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:46.123734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:46.123835 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:46.123912 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:46.125779 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.126914 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.129408 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.130026 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.130135 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:46.138135 (+ 8000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:46.138149 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:46.138168 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:46.138214 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:46.142848 (+ 4634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:46.143652 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:46.143659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:46.143661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:46.143808 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:46.144246 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:46.144249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:46.144251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:46.144588 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:46.144936 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:46.145264 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 23:03:46.145489 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 23:03:46.146641 13505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:46.149654 13872 ts_tablet_manager.cc:1918] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:46.149845 13872 log.cc:1196] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/cb2b2309a4df4099a0562c42ddf86da4
I20260809 23:03:46.150395 13872 ts_tablet_manager.cc:1939] T cb2b2309a4df4099a0562c42ddf86da4 P 5ea10d8d50274dcfad62680668d14d75: Deleting consensus metadata
I20260809 23:03:46.152052 13262 catalog_manager.cc:4841] TS 5ea10d8d50274dcfad62680668d14d75 (127.24.31.194:44693): tablet cb2b2309a4df4099a0562c42ddf86da4 (table test-table [id=ff8a116cf8c341cab640d0f81b6b63f2]) successfully deleted
I20260809 23:03:46.252255 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 13355
I20260809 23:03:46.274104 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 13516
Aug 09 23:03:46 dist-test-slave-vxj2 krb5kdc[13228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 23:03:46.292379 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:46.275131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56316 (local address 127.24.31.194:44693)
0809 23:03:46.275313 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:46.275318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:46.275336 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:03:46.275421 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:46.275424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:46.275495 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:46.275597 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:46.277311 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.278144 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.280072 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.280526 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.280621 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:46.285982 (+ 5361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:46.285997 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:46.286017 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:46.286051 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:46.289344 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:46.290086 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:46.290094 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:46.290096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:46.290264 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:46.290678 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:46.290684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:46.290687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:46.291083 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:46.291456 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:46.291740 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 23:03:46.291981 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 23:03:46.293226 13662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:46.410195 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 13516
I20260809 23:03:46.433048 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 13675
Aug 09 23:03:46 dist-test-slave-vxj2 krb5kdc[13228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 23:03:46.449445 13846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:46.433912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39340 (local address 127.24.31.195:46629)
0809 23:03:46.434116 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:46.434121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:46.434142 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:03:46.434329 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:46.434334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:46.434410 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:46.434495 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:46.436013 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.436707 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.438270 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:46.438632 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:46.438726 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:46.443509 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:46.443523 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:46.443539 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:46.443572 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:46.446548 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:46.447218 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:46.447226 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:46.447228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:46.447365 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:46.447822 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:46.447827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:46.447829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:46.448202 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:46.448505 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:46.448801 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 23:03:46.448977 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 23:03:46.450141 13817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:46.541909 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 13675
I20260809 23:03:46.561669 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 13244
I20260809 23:03:46.562929 13319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:46.675098 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 13244
2026-08-09T23:03:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4590 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T23:03:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:03:46Z Disabled control of system clock
I20260809 23:03:46.762015 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42389
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:41031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:42389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:03:47.101024 13883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:47.101374 13883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:47.101475 13883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:47.108657 13883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:03:47.108764 13883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:47.108803 13883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:47.108855 13883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:03:47.108899 13883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:03:47.118058 13883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:42389
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:47.119287 13883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:47.121119 13883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:47.133180 13889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:47.133211 13890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:47.133109 13892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:47.134735 13883 server_base.cc:1034] running on GCE node
I20260809 23:03:47.135470 13883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:47.136822 13883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:47.138074 13883 hybrid_clock.cc:648] HybridClock initialized: now 1786316627138030 us; error 37 us; skew 500 ppm
I20260809 23:03:47.138427 13883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:47.140774 13883 webserver.cc:460] Webserver started at http://127.24.31.254:33791/ using document root <none> and password file <none>
I20260809 23:03:47.141557 13883 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:47.141685 13883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:47.142010 13883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:47.144905 13883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "d3943a20783b47acadab995da5f4c320"
format_stamp: "Formatted at 2026-08-09 23:03:47 on dist-test-slave-vxj2"
server_key: "c7c749b0e75d6d1ab8b8742eb6f0e65a"
server_key_iv: "35c99b891119582ffff34a891d05dee9"
server_key_version: "encryptionkey@0"
I20260809 23:03:47.145828 13883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "d3943a20783b47acadab995da5f4c320"
format_stamp: "Formatted at 2026-08-09 23:03:47 on dist-test-slave-vxj2"
server_key: "c7c749b0e75d6d1ab8b8742eb6f0e65a"
server_key_iv: "35c99b891119582ffff34a891d05dee9"
server_key_version: "encryptionkey@0"
I20260809 23:03:47.151188 13883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:03:47.154662 13899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:47.156203 13883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:03:47.156418 13883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "d3943a20783b47acadab995da5f4c320"
format_stamp: "Formatted at 2026-08-09 23:03:47 on dist-test-slave-vxj2"
server_key: "c7c749b0e75d6d1ab8b8742eb6f0e65a"
server_key_iv: "35c99b891119582ffff34a891d05dee9"
server_key_version: "encryptionkey@0"
I20260809 23:03:47.156643 13883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:47.167644 13883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:47.168556 13883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:47.168859 13883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:47.188802 13883 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:42389
I20260809 23:03:47.188885 13959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:42389 every 8 connection(s)
I20260809 23:03:47.190184 13883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:03:47.194276 13960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:47.198928 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 13883
I20260809 23:03:47.199139 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:03:47.199608 24703 external_mini_cluster.cc:1428] Setting key eded639acd77473092925e049cdacc70
I20260809 23:03:47.207567 13960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320: Bootstrap starting.
I20260809 23:03:47.211563 13960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:47.212831 13960 log.cc:824] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:47.216291 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:47.201575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46174 (local address 127.24.31.254:42389)
0809 23:03:47.202686 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:47.202702 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:47.202763 (+ 61us) server_negotiation.cc:407] Connection header received
0809 23:03:47.205597 (+ 2834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:47.205627 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:47.206187 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:47.206653 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:47.207232 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:47.209304 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:47.210720 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:47.211859 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:47.212103 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:47.213464 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:47.213482 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:47.213506 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:47.214151 (+ 645us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:47.214294 (+ 143us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:47.214325 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:47.214465 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:47.214684 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 23:03:47.215010 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":350,"threads_started":1}
I20260809 23:03:47.216401 13960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320: No bootstrap required, opened a new log
I20260809 23:03:47.224788 13960 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3943a20783b47acadab995da5f4c320" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42389 } }
I20260809 23:03:47.225134 13960 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:47.225296 13960 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3943a20783b47acadab995da5f4c320, State: Initialized, Role: FOLLOWER
I20260809 23:03:47.225796 13960 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3943a20783b47acadab995da5f4c320" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42389 } }
I20260809 23:03:47.226120 13960 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:03:47.226322 13960 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:03:47.226588 13960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:47.228904 13960 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3943a20783b47acadab995da5f4c320" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42389 } }
I20260809 23:03:47.229509 13960 leader_election.cc:302] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3943a20783b47acadab995da5f4c320; no voters:
I20260809 23:03:47.230134 13960 leader_election.cc:288] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:03:47.230293 13966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:47.231065 13966 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [term 1 LEADER]: Becoming Leader. State: Replica: d3943a20783b47acadab995da5f4c320, State: Running, Role: LEADER
I20260809 23:03:47.231688 13966 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3943a20783b47acadab995da5f4c320" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42389 } }
I20260809 23:03:47.232342 13960 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:03:47.235678 13967 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3943a20783b47acadab995da5f4c320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3943a20783b47acadab995da5f4c320" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42389 } } }
I20260809 23:03:47.236084 13967 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:47.236940 13974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:03:47.237287 13968 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3943a20783b47acadab995da5f4c320. Latest consensus state: current_term: 1 leader_uuid: "d3943a20783b47acadab995da5f4c320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3943a20783b47acadab995da5f4c320" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42389 } } }
I20260809 23:03:47.237730 13968 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320 [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:47.243158 13974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:03:47.251986 13974 catalog_manager.cc:1347] Generated new cluster ID: a4fb9429c11646fdbb741ec00748eb15
I20260809 23:03:47.252113 13974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:03:47.263087 13974 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:03:47.264927 13974 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:03:47.274991 13974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320: Generated new TSK 0
I20260809 23:03:47.276068 13974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:03:47.284747 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42389
--builtin_ntp_servers=127.24.31.212:41031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:03:47.626230 13987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:47.626592 13987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:47.626761 13987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:47.634131 13987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:47.634280 13987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:47.634465 13987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:03:47.643358 13987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:47.644596 13987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:47.646411 13987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:47.659709 13994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:47.659816 13995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:47.659819 13997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:47.661010 13987 server_base.cc:1034] running on GCE node
I20260809 23:03:47.662017 13987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:47.663043 13987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:47.664317 13987 hybrid_clock.cc:648] HybridClock initialized: now 1786316627664266 us; error 32 us; skew 500 ppm
I20260809 23:03:47.664661 13987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:47.667205 13987 webserver.cc:460] Webserver started at http://127.24.31.193:37205/ using document root <none> and password file <none>
I20260809 23:03:47.667937 13987 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:47.668087 13987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:47.668457 13987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:47.671486 13987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "857896783d164282b265c0e6f07e2972"
format_stamp: "Formatted at 2026-08-09 23:03:47 on dist-test-slave-vxj2"
server_key: "e30c5d7b5a9b9c983463825ed141ee99"
server_key_iv: "727998b7478d351d540dab28527f71e7"
server_key_version: "encryptionkey@0"
I20260809 23:03:47.672360 13987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "857896783d164282b265c0e6f07e2972"
format_stamp: "Formatted at 2026-08-09 23:03:47 on dist-test-slave-vxj2"
server_key: "e30c5d7b5a9b9c983463825ed141ee99"
server_key_iv: "727998b7478d351d540dab28527f71e7"
server_key_version: "encryptionkey@0"
I20260809 23:03:47.677733 13987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 23:03:47.681231 14003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:47.682554 13987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:03:47.682760 13987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "857896783d164282b265c0e6f07e2972"
format_stamp: "Formatted at 2026-08-09 23:03:47 on dist-test-slave-vxj2"
server_key: "e30c5d7b5a9b9c983463825ed141ee99"
server_key_iv: "727998b7478d351d540dab28527f71e7"
server_key_version: "encryptionkey@0"
I20260809 23:03:47.683219 13987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:47.705968 13987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:47.706785 13987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:47.707096 13987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:47.708033 13987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:47.709520 13987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:47.709642 13987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:47.709784 13987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:47.709877 13987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:47.743656 13987 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:37143
I20260809 23:03:47.743744 14131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:37143 every 8 connection(s)
I20260809 23:03:47.745023 13987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:03:47.745563 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 13987
I20260809 23:03:47.745779 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:03:47.746387 24703 external_mini_cluster.cc:1428] Setting key c926775170b1b6b21e49a874fb6bc4b3
I20260809 23:03:47.749950 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42389
--builtin_ntp_servers=127.24.31.212:41031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:03:47.760833 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:47.748066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:52287 (local address 127.24.31.254:42389)
0809 23:03:47.748577 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:47.748584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:47.753071 (+ 4487us) server_negotiation.cc:407] Connection header received
0809 23:03:47.753141 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:47.753167 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:47.753274 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:47.753386 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:47.754612 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:47.755746 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:47.757511 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:47.758261 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:47.758407 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:47.759731 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:47.759739 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:47.759743 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:47.759769 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:47.759825 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:47.759844 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:47.759936 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:47.760222 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 23:03:47.760365 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":246,"threads_started":1}
I20260809 23:03:47.762154 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:47.748760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:42389 (local address 127.24.31.193:52287)
0809 23:03:47.749977 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:47.750037 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:47.752743 (+ 2706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:47.753691 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:47.753700 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:47.753806 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:47.754417 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:47.754437 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:47.755937 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:47.755941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:47.757363 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:47.757378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:47.758479 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:47.758483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:47.758683 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:47.759432 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:03:47.759435 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:47.759473 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:03:47.759554 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:47.759969 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:47.759987 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:47.759994 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:03:47.760201 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 23:03:47.760636 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":352,"threads_started":1}
I20260809 23:03:47.763703 14132 heartbeater.cc:347] Connected to a master server at 127.24.31.254:42389
I20260809 23:03:47.764025 14132 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:47.764747 14132 heartbeater.cc:511] Master 127.24.31.254:42389 requested a full tablet report, sending...
I20260809 23:03:47.766846 13920 ts_manager.cc:194] Registered new tserver with Master: 857896783d164282b265c0e6f07e2972 (127.24.31.193:37143)
I20260809 23:03:47.768363 13920 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.193:52287
W20260809 23:03:48.084203 14139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:48.084553 14139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:48.084725 14139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:48.092286 14139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:48.092428 14139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:48.092696 14139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:03:48.101675 14139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:48.103019 14139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:48.104853 14139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:48.117787 14146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:48.117910 14145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:48.118089 14148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:48.118887 14139 server_base.cc:1034] running on GCE node
I20260809 23:03:48.119506 14139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:48.120424 14139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:48.121631 14139 hybrid_clock.cc:648] HybridClock initialized: now 1786316628121564 us; error 51 us; skew 500 ppm
I20260809 23:03:48.122020 14139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:48.124220 14139 webserver.cc:460] Webserver started at http://127.24.31.194:34165/ using document root <none> and password file <none>
I20260809 23:03:48.124903 14139 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:48.125020 14139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:48.125396 14139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:48.128213 14139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "25bd61bbcff24970a7973cce61ede14a"
format_stamp: "Formatted at 2026-08-09 23:03:48 on dist-test-slave-vxj2"
server_key: "8b00b151f48448d25eb71bcbe12a6980"
server_key_iv: "d5d6d8acab4ca75786fd3131fd68361a"
server_key_version: "encryptionkey@0"
I20260809 23:03:48.129017 14139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "25bd61bbcff24970a7973cce61ede14a"
format_stamp: "Formatted at 2026-08-09 23:03:48 on dist-test-slave-vxj2"
server_key: "8b00b151f48448d25eb71bcbe12a6980"
server_key_iv: "d5d6d8acab4ca75786fd3131fd68361a"
server_key_version: "encryptionkey@0"
I20260809 23:03:48.134431 14139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 23:03:48.137962 14154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:48.139284 14139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:03:48.139477 14139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "25bd61bbcff24970a7973cce61ede14a"
format_stamp: "Formatted at 2026-08-09 23:03:48 on dist-test-slave-vxj2"
server_key: "8b00b151f48448d25eb71bcbe12a6980"
server_key_iv: "d5d6d8acab4ca75786fd3131fd68361a"
server_key_version: "encryptionkey@0"
I20260809 23:03:48.139757 14139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:48.168226 14139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:48.169101 14139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:48.169556 14139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:48.170601 14139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:48.172122 14139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:48.172237 14139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:48.172353 14139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:48.172456 14139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:48.208204 14139 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:39359
I20260809 23:03:48.208339 14283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:39359 every 8 connection(s)
I20260809 23:03:48.209795 14139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:03:48.210963 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 14139
I20260809 23:03:48.211189 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:03:48.211836 24703 external_mini_cluster.cc:1428] Setting key a12a9b7bdeae62f8749d31e1cb0043aa
I20260809 23:03:48.216043 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42389
--builtin_ntp_servers=127.24.31.212:41031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:03:48.235862 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.214320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:47969 (local address 127.24.31.254:42389)
0809 23:03:48.214588 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:48.214597 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:48.224724 (+ 10127us) server_negotiation.cc:407] Connection header received
0809 23:03:48.227917 (+ 3193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:48.227921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:48.227996 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:48.228095 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:48.229682 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.230561 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.232447 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.233266 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.233409 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.234941 (+ 1532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:48.234950 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:48.234957 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:48.234984 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:48.235030 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:48.235048 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:48.235142 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:48.235355 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 23:03:48.235468 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 23:03:48.236971 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.213785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:42389 (local address 127.24.31.194:47969)
0809 23:03:48.224309 (+ 10524us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:48.224367 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:48.227547 (+ 3180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:48.228420 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:48.228436 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:48.228560 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:48.229475 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.229513 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.230739 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.230745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.232280 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.232294 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.233507 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.233513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.233813 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.234613 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:03:48.234615 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:48.234662 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:03:48.234741 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:48.235147 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:48.235155 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:48.235161 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:03:48.235326 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 23:03:48.235625 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9872,"thread_start_us":10883,"threads_started":1}
I20260809 23:03:48.238634 14285 heartbeater.cc:347] Connected to a master server at 127.24.31.254:42389
I20260809 23:03:48.238961 14285 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:48.239732 14285 heartbeater.cc:511] Master 127.24.31.254:42389 requested a full tablet report, sending...
I20260809 23:03:48.241398 13920 ts_manager.cc:194] Registered new tserver with Master: 25bd61bbcff24970a7973cce61ede14a (127.24.31.194:39359)
I20260809 23:03:48.242699 13920 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.194:47969
W20260809 23:03:48.556598 14288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:48.556872 14288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:48.556960 14288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:48.564345 14288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:48.564443 14288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:48.564554 14288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:03:48.573724 14288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:42389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:48.574970 14288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:48.576690 14288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:48.589828 14300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:48.589851 14297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:48.590144 14298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:48.590711 14288 server_base.cc:1034] running on GCE node
I20260809 23:03:48.591334 14288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:48.592289 14288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:48.593532 14288 hybrid_clock.cc:648] HybridClock initialized: now 1786316628593489 us; error 28 us; skew 500 ppm
I20260809 23:03:48.593869 14288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:48.596216 14288 webserver.cc:460] Webserver started at http://127.24.31.195:37567/ using document root <none> and password file <none>
I20260809 23:03:48.596931 14288 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:48.597067 14288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:48.597419 14288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:48.600275 14288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "69d19ac6862b4e0e956ab7b867a406c0"
format_stamp: "Formatted at 2026-08-09 23:03:48 on dist-test-slave-vxj2"
server_key: "882ca4c42bcb96ecae60fab6b6409827"
server_key_iv: "b83d0d360f5664c617b3aab64b3a8ca4"
server_key_version: "encryptionkey@0"
I20260809 23:03:48.601071 14288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "69d19ac6862b4e0e956ab7b867a406c0"
format_stamp: "Formatted at 2026-08-09 23:03:48 on dist-test-slave-vxj2"
server_key: "882ca4c42bcb96ecae60fab6b6409827"
server_key_iv: "b83d0d360f5664c617b3aab64b3a8ca4"
server_key_version: "encryptionkey@0"
I20260809 23:03:48.606271 14288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 23:03:48.609982 14306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:48.611379 14288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:03:48.611557 14288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "69d19ac6862b4e0e956ab7b867a406c0"
format_stamp: "Formatted at 2026-08-09 23:03:48 on dist-test-slave-vxj2"
server_key: "882ca4c42bcb96ecae60fab6b6409827"
server_key_iv: "b83d0d360f5664c617b3aab64b3a8ca4"
server_key_version: "encryptionkey@0"
I20260809 23:03:48.611774 14288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:48.651559 14288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:48.652426 14288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:48.652740 14288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:48.653810 14288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:48.655267 14288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:48.655390 14288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:48.655515 14288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:48.655602 14288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:48.690423 14288 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:42693
I20260809 23:03:48.690517 14439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:42693 every 8 connection(s)
I20260809 23:03:48.691730 14288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:03:48.692303 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 14288
I20260809 23:03:48.692519 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:03:48.693274 24703 external_mini_cluster.cc:1428] Setting key a2068eee01e1bcc6844ad09c9c6ab20d
I20260809 23:03:48.705318 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.694926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:42719 (local address 127.24.31.254:42389)
0809 23:03:48.695122 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:48.695127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:48.696523 (+ 1396us) server_negotiation.cc:407] Connection header received
0809 23:03:48.698834 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:48.698839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:48.698932 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:48.699042 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:48.700110 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.700870 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.702519 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.702953 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.703090 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.704409 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:48.704414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:48.704417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:48.704437 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:48.704469 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:48.704480 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:48.704592 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:48.704855 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 23:03:48.704957 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 23:03:48.706333 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.695315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:42389 (local address 127.24.31.195:42719)
0809 23:03:48.696354 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:48.696411 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:48.698555 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:48.699278 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:48.699287 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:48.699381 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:48.699944 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.699960 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.701038 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.701045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.702370 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.702382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.703191 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.703199 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.703437 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.704082 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:03:48.704085 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:48.704123 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:03:48.704201 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:48.704607 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:48.704619 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:48.704627 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 23:03:48.704844 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 23:03:48.705257 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":470,"threads_started":1}
I20260809 23:03:48.707844 14440 heartbeater.cc:347] Connected to a master server at 127.24.31.254:42389
I20260809 23:03:48.708143 14440 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:48.709026 14440 heartbeater.cc:511] Master 127.24.31.254:42389 requested a full tablet report, sending...
I20260809 23:03:48.710510 13918 ts_manager.cc:194] Registered new tserver with Master: 69d19ac6862b4e0e956ab7b867a406c0 (127.24.31.195:42693)
I20260809 23:03:48.711346 13918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.195:42719
I20260809 23:03:48.722883 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:03:48.733281 14135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.727108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.24.31.254:42389)
0809 23:03:48.727352 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:48.727357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:48.727822 (+ 465us) server_negotiation.cc:407] Connection header received
0809 23:03:48.727930 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:48.727934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:48.728013 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:48.728100 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:48.728601 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.729406 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.730821 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.731217 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.731317 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.732497 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:48.732502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:48.732505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:48.732521 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:48.732552 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:48.732563 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:48.732670 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:48.732833 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 23:03:48.732926 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 23:03:48.737468 13918 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:03:48.745119 13918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:03:48.768651 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.758542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.24.31.193:37143)
0809 23:03:48.759387 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:48.759395 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:48.760132 (+ 737us) server_negotiation.cc:407] Connection header received
0809 23:03:48.760265 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:48.760276 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:48.760557 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:48.761399 (+ 842us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:48.761870 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.763441 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.766446 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.767476 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.767640 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.767940 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:48.768109 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 23:03:48.768274 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":1209,"threads_started":1}
I20260809 23:03:48.768709 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.758228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37143 (local address 127.0.0.1:38510)
0809 23:03:48.759508 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:48.759588 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:48.759790 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:48.760954 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:48.760962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:48.761045 (+ 83us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:48.761622 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.761639 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.763593 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.763599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.765925 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.765942 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.767675 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.767681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.767863 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.767917 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 23:03:48.768069 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 23:03:48.768259 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":267,"thread_start_us":742,"threads_started":1}
I20260809 23:03:48.769524 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.758147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.24.31.194:39359)
0809 23:03:48.758731 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:48.758743 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:48.760318 (+ 1575us) server_negotiation.cc:407] Connection header received
0809 23:03:48.760480 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:48.760488 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:48.760710 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:48.760916 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:48.762413 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.763789 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.766522 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.767981 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.768087 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.768321 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:48.768972 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 23:03:48.769139 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":299,"threads_started":1}
I20260809 23:03:48.769521 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.759623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:42693 (local address 127.0.0.1:51460)
0809 23:03:48.760688 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:48.760709 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:48.760831 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:48.761662 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:48.761666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:48.761693 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:48.762454 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.762468 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.764283 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.764292 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.766608 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.766623 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.768692 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.768701 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.768916 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.768943 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 23:03:48.769020 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 23:03:48.769138 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":2,"thread_start_us":797,"threads_started":1}
I20260809 23:03:48.770166 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.757935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:39359 (local address 127.0.0.1:34778)
0809 23:03:48.760117 (+ 2182us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:48.760178 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:48.760328 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:48.761195 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:48.761206 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:48.761228 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:48.762184 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.762206 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.763919 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.763927 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.766280 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.766295 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.768501 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.768507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.768757 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.768793 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 23:03:48.769722 (+ 929us) client_negotiation.cc:240] Negotiation successful
0809 23:03:48.769858 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"mutex_wait_us":35,"thread_start_us":837,"threads_started":1}
I20260809 23:03:48.770514 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.759990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51460 (local address 127.24.31.195:42693)
0809 23:03:48.761045 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:48.761054 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:48.761085 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:03:48.761202 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:48.761209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:48.761412 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:48.761596 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:48.762657 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.764134 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.766828 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.768191 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.768311 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.768563 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:48.769940 (+ 1377us) server_negotiation.cc:299] Negotiation successful
0809 23:03:48.770107 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":945,"threads_started":1}
I20260809 23:03:48.771355 14132 heartbeater.cc:503] Master 127.24.31.254:42389 was elected leader, sending a full tablet report...
I20260809 23:03:48.773273 14059 tablet_service.cc:1467] Processing CreateTablet for tablet b9ab9bf85e304698ab87379b831d6275 (DEFAULT_TABLE table=test-table [id=864a8458a7754653845e40896f8af0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:48.773932 14367 tablet_service.cc:1467] Processing CreateTablet for tablet b9ab9bf85e304698ab87379b831d6275 (DEFAULT_TABLE table=test-table [id=864a8458a7754653845e40896f8af0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:48.773936 14210 tablet_service.cc:1467] Processing CreateTablet for tablet b9ab9bf85e304698ab87379b831d6275 (DEFAULT_TABLE table=test-table [id=864a8458a7754653845e40896f8af0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:48.774627 14059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9ab9bf85e304698ab87379b831d6275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:48.775271 14367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9ab9bf85e304698ab87379b831d6275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:48.775270 14210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9ab9bf85e304698ab87379b831d6275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:48.786666 14462 tablet_bootstrap.cc:492] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: Bootstrap starting.
I20260809 23:03:48.790555 14462 tablet_bootstrap.cc:654] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:48.790928 14463 tablet_bootstrap.cc:492] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: Bootstrap starting.
I20260809 23:03:48.792034 14462 log.cc:824] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:48.792621 14464 tablet_bootstrap.cc:492] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: Bootstrap starting.
I20260809 23:03:48.794780 14463 tablet_bootstrap.cc:654] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:48.795682 14464 tablet_bootstrap.cc:654] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:48.795979 14463 log.cc:824] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:48.796064 14462 tablet_bootstrap.cc:492] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: No bootstrap required, opened a new log
I20260809 23:03:48.796315 14462 ts_tablet_manager.cc:1397] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 23:03:48.796782 14464 log.cc:824] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:48.799055 14463 tablet_bootstrap.cc:492] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: No bootstrap required, opened a new log
I20260809 23:03:48.799348 14463 ts_tablet_manager.cc:1397] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 23:03:48.799710 14464 tablet_bootstrap.cc:492] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: No bootstrap required, opened a new log
I20260809 23:03:48.799960 14464 ts_tablet_manager.cc:1397] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 23:03:48.806069 14463 raft_consensus.cc:348] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } }
I20260809 23:03:48.806483 14463 raft_consensus.cc:374] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:48.806612 14463 raft_consensus.cc:729] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69d19ac6862b4e0e956ab7b867a406c0, State: Initialized, Role: FOLLOWER
I20260809 23:03:48.806949 14462 raft_consensus.cc:348] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } }
I20260809 23:03:48.807348 14462 raft_consensus.cc:374] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:48.807233 14463 consensus_queue.cc:260] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } }
I20260809 23:03:48.807451 14462 raft_consensus.cc:729] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 857896783d164282b265c0e6f07e2972, State: Initialized, Role: FOLLOWER
I20260809 23:03:48.807998 14462 consensus_queue.cc:260] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } }
I20260809 23:03:48.808550 14440 heartbeater.cc:503] Master 127.24.31.254:42389 was elected leader, sending a full tablet report...
I20260809 23:03:48.809186 14463 ts_tablet_manager.cc:1428] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: Time spent starting tablet: real 0.010s user 0.003s sys 0.006s
I20260809 23:03:48.811076 14462 ts_tablet_manager.cc:1428] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 23:03:48.813313 14464 raft_consensus.cc:348] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } }
I20260809 23:03:48.813755 14464 raft_consensus.cc:374] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:48.813884 14464 raft_consensus.cc:729] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25bd61bbcff24970a7973cce61ede14a, State: Initialized, Role: FOLLOWER
I20260809 23:03:48.814508 14464 consensus_queue.cc:260] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } }
I20260809 23:03:48.815685 14285 heartbeater.cc:503] Master 127.24.31.254:42389 was elected leader, sending a full tablet report...
I20260809 23:03:48.816293 14464 ts_tablet_manager.cc:1428] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260809 23:03:48.903717 14470 raft_consensus.cc:482] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:03:48.904120 14470 raft_consensus.cc:504] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } }
I20260809 23:03:48.905860 14470 leader_election.cc:288] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25bd61bbcff24970a7973cce61ede14a (127.24.31.194:39359), 69d19ac6862b4e0e956ab7b867a406c0 (127.24.31.195:42693)
I20260809 23:03:48.913108 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.906360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:46013 (local address 127.24.31.194:39359)
0809 23:03:48.906627 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:48.906636 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:48.907015 (+ 379us) server_negotiation.cc:407] Connection header received
0809 23:03:48.907290 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:48.907297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:48.907403 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:48.907516 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:48.908188 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.909119 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.911196 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.912123 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.912298 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.912369 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:48.912720 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 23:03:48.912845 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 23:03:48.913412 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.907144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:36391 (local address 127.24.31.195:42693)
0809 23:03:48.907396 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:48.907406 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:48.907858 (+ 452us) server_negotiation.cc:407] Connection header received
0809 23:03:48.907968 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:48.907975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:48.908052 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:48.908143 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:48.908770 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.909694 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.911587 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.912504 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.912677 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.912743 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:48.913019 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 23:03:48.913139 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 23:03:48.913484 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.906938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:42693 (local address 127.24.31.193:36391)
0809 23:03:48.907746 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:48.907761 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:48.907860 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:48.908178 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:48.908182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:48.908206 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:48.908605 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.908616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.909843 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.909851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.911396 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.911410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.912691 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.912697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.912869 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.912892 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 23:03:48.912993 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 23:03:48.913115 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":24,"thread_start_us":441,"threads_started":1}
I20260809 23:03:48.914088 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.906157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:39359 (local address 127.24.31.193:46013)
0809 23:03:48.906882 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:48.906908 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:48.907133 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:48.907553 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:48.907561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:48.907601 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:48.908011 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.908026 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.909301 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.909307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.911007 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:48.911022 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.912319 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.912325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:48.912512 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.912543 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 23:03:48.913599 (+ 1056us) client_negotiation.cc:240] Negotiation successful
0809 23:03:48.913736 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":22,"thread_start_us":381,"threads_started":1}
I20260809 23:03:48.914666 14390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ab9bf85e304698ab87379b831d6275" candidate_uuid: "857896783d164282b265c0e6f07e2972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69d19ac6862b4e0e956ab7b867a406c0" is_pre_election: true
I20260809 23:03:48.915165 14233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ab9bf85e304698ab87379b831d6275" candidate_uuid: "857896783d164282b265c0e6f07e2972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bd61bbcff24970a7973cce61ede14a" is_pre_election: true
I20260809 23:03:48.915339 14390 raft_consensus.cc:2454] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 857896783d164282b265c0e6f07e2972 in term 0.
I20260809 23:03:48.915759 14233 raft_consensus.cc:2454] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 857896783d164282b265c0e6f07e2972 in term 0.
I20260809 23:03:48.916040 14008 leader_election.cc:302] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69d19ac6862b4e0e956ab7b867a406c0, 857896783d164282b265c0e6f07e2972; no voters:
I20260809 23:03:48.916503 14470 raft_consensus.cc:2781] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:03:48.916664 14470 raft_consensus.cc:482] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:03:48.916850 14470 raft_consensus.cc:3037] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:48.919072 14470 raft_consensus.cc:504] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } }
I20260809 23:03:48.919684 14470 leader_election.cc:288] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [CANDIDATE]: Term 1 election: Requested vote from peers 25bd61bbcff24970a7973cce61ede14a (127.24.31.194:39359), 69d19ac6862b4e0e956ab7b867a406c0 (127.24.31.195:42693)
I20260809 23:03:48.920077 14233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ab9bf85e304698ab87379b831d6275" candidate_uuid: "857896783d164282b265c0e6f07e2972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25bd61bbcff24970a7973cce61ede14a"
I20260809 23:03:48.920153 14390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ab9bf85e304698ab87379b831d6275" candidate_uuid: "857896783d164282b265c0e6f07e2972" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69d19ac6862b4e0e956ab7b867a406c0"
I20260809 23:03:48.920336 14233 raft_consensus.cc:3037] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:48.920356 14390 raft_consensus.cc:3037] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:48.922626 14390 raft_consensus.cc:2454] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 857896783d164282b265c0e6f07e2972 in term 1.
I20260809 23:03:48.922662 14233 raft_consensus.cc:2454] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 857896783d164282b265c0e6f07e2972 in term 1.
I20260809 23:03:48.923085 14008 leader_election.cc:302] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25bd61bbcff24970a7973cce61ede14a, 857896783d164282b265c0e6f07e2972; no voters:
I20260809 23:03:48.923418 14470 raft_consensus.cc:2781] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:48.923977 14470 raft_consensus.cc:686] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 1 LEADER]: Becoming Leader. State: Replica: 857896783d164282b265c0e6f07e2972, State: Running, Role: LEADER
I20260809 23:03:48.924590 14470 consensus_queue.cc:237] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } }
I20260809 23:03:48.929585 13918 catalog_manager.cc:5495] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 reported cstate change: term changed from 0 to 1, leader changed from <none> to 857896783d164282b265c0e6f07e2972 (127.24.31.193). New cstate: current_term: 1 leader_uuid: "857896783d164282b265c0e6f07e2972" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "857896783d164282b265c0e6f07e2972" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 37143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 } health_report { overall_health: UNKNOWN } } }
W20260809 23:03:48.944247 14441 tablet.cc:2369] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:48.963979 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:48.958061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.24.31.193:37143)
0809 23:03:48.958299 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:48.958308 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:48.958330 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:03:48.958589 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:48.958593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:48.958684 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:48.958860 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:03:48.959272 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.959980 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.961688 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:48.962114 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:48.962269 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:48.962876 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:03:48.963366 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:03:48.963621 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:48.963706 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 23:03:48.963749 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 23:03:48.972409 14233 raft_consensus.cc:1261] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [term 1 FOLLOWER]: Refusing update from remote peer 857896783d164282b265c0e6f07e2972: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:48.972406 14390 raft_consensus.cc:1261] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [term 1 FOLLOWER]: Refusing update from remote peer 857896783d164282b265c0e6f07e2972: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:48.973487 14476 consensus_queue.cc:1035] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25bd61bbcff24970a7973cce61ede14a" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 39359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:48.973881 14470 consensus_queue.cc:1035] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69d19ac6862b4e0e956ab7b867a406c0" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 42693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:03:48.977537 14286 tablet.cc:2369] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:48.987396 14482 mvcc.cc:205] Tried to move back new op lower bound from 7316752912260182016 to 7316752912080388096. Current Snapshot: MvccSnapshot[applied={T|T < 7316752912260182016}]
I20260809 23:03:48.987692 14481 mvcc.cc:205] Tried to move back new op lower bound from 7316752912260182016 to 7316752912080388096. Current Snapshot: MvccSnapshot[applied={T|T < 7316752912260182016}]
I20260809 23:03:49.002347 13918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:49.002643 13918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:49.006520 13918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3943a20783b47acadab995da5f4c320: Sending DeleteTablet for 3 replicas of tablet b9ab9bf85e304698ab87379b831d6275
I20260809 23:03:49.007736 14210 tablet_service.cc:1514] Processing DeleteTablet for tablet b9ab9bf85e304698ab87379b831d6275 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:48 UTC) from {username='slave'} at 127.0.0.1:34778
I20260809 23:03:49.007748 14059 tablet_service.cc:1514] Processing DeleteTablet for tablet b9ab9bf85e304698ab87379b831d6275 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:48 UTC) from {username='slave'} at 127.0.0.1:38510
I20260809 23:03:49.007870 14367 tablet_service.cc:1514] Processing DeleteTablet for tablet b9ab9bf85e304698ab87379b831d6275 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:48 UTC) from {username='slave'} at 127.0.0.1:51460
I20260809 23:03:49.011942 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 13987
I20260809 23:03:49.020440 14492 tablet_replica.cc:330] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: stopping tablet replica
I20260809 23:03:49.020668 14493 tablet_replica.cc:330] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: stopping tablet replica
I20260809 23:03:49.020934 14492 raft_consensus.cc:2229] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:49.021097 14493 raft_consensus.cc:2229] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:03:49.021626 14492 raft_consensus.cc:2258] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:49.021826 14493 raft_consensus.cc:2258] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:49.023291 14494 tablet_replica.cc:330] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: stopping tablet replica
I20260809 23:03:49.023715 14494 raft_consensus.cc:2229] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:49.024174 14492 ts_tablet_manager.cc:1905] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:49.024246 14494 raft_consensus.cc:2258] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:49.025076 14493 ts_tablet_manager.cc:1905] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:49.026829 14494 ts_tablet_manager.cc:1905] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:49.031445 14492 ts_tablet_manager.cc:1918] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:49.031672 14492 log.cc:1196] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/b9ab9bf85e304698ab87379b831d6275
I20260809 23:03:49.032331 14492 ts_tablet_manager.cc:1939] T b9ab9bf85e304698ab87379b831d6275 P 25bd61bbcff24970a7973cce61ede14a: Deleting consensus metadata
I20260809 23:03:49.034473 13902 catalog_manager.cc:4841] TS 25bd61bbcff24970a7973cce61ede14a (127.24.31.194:39359): tablet b9ab9bf85e304698ab87379b831d6275 (table test-table [id=864a8458a7754653845e40896f8af0b9]) successfully deleted
I20260809 23:03:49.035072 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.013398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.24.31.193:37143)
0809 23:03:49.013594 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:49.013600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:49.022570 (+ 8970us) server_negotiation.cc:407] Connection header received
0809 23:03:49.022803 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:49.022811 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:49.022938 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:49.023048 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:49.023729 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:49.024871 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:49.026854 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:49.027510 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:49.027611 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:49.033113 (+ 5502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:49.033124 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:49.033144 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:49.034004 (+ 860us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:49.034100 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:49.034125 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:49.034228 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:49.034531 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 23:03:49.034643 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 23:03:49.035903 14494 ts_tablet_manager.cc:1918] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:49.036113 14494 log.cc:1196] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/b9ab9bf85e304698ab87379b831d6275
I20260809 23:03:49.036679 14126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:49.036760 14494 ts_tablet_manager.cc:1939] T b9ab9bf85e304698ab87379b831d6275 P 69d19ac6862b4e0e956ab7b867a406c0: Deleting consensus metadata
I20260809 23:03:49.039072 13902 catalog_manager.cc:4841] TS 69d19ac6862b4e0e956ab7b867a406c0 (127.24.31.195:42693): tablet b9ab9bf85e304698ab87379b831d6275 (table test-table [id=864a8458a7754653845e40896f8af0b9]) successfully deleted
I20260809 23:03:49.135031 14493 ts_tablet_manager.cc:1918] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:49.135341 14493 log.cc:1196] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/b9ab9bf85e304698ab87379b831d6275
I20260809 23:03:49.135955 14493 ts_tablet_manager.cc:1939] T b9ab9bf85e304698ab87379b831d6275 P 857896783d164282b265c0e6f07e2972: Deleting consensus metadata
I20260809 23:03:49.136759 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 13987
I20260809 23:03:49.158329 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 14139
W20260809 23:03:49.158289 13900 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.31.193:37143 (error 108)
I20260809 23:03:49.160214 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.159771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37143 (local address 127.0.0.1:38520)
0809 23:03:49.159928 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:49.160028 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:37143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 23:03:49.160511 13900 catalog_manager.cc:4584] TS 857896783d164282b265c0e6f07e2972 (127.24.31.193:37143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9ab9bf85e304698ab87379b831d6275: Network error: Client connection negotiation failed: client connection to 127.24.31.193:37143: connect: Connection refused (error 111)
I20260809 23:03:49.166030 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.159106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.24.31.194:39359)
0809 23:03:49.159306 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:49.159311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:49.159485 (+ 174us) server_negotiation.cc:407] Connection header received
0809 23:03:49.159730 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:49.159738 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:49.160007 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:49.160143 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:49.160666 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:49.161730 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:49.162942 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:49.163372 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:49.163482 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:49.164448 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:49.164458 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:49.164472 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:49.165128 (+ 656us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:49.165264 (+ 136us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:49.165290 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:49.165411 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:49.165527 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 23:03:49.165673 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 23:03:49.166785 14278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:49.214293 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.213718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37143 (local address 127.0.0.1:38522)
0809 23:03:49.213959 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:49.214076 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:37143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 23:03:49.215489 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.215004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37143 (local address 127.0.0.1:38534)
0809 23:03:49.215221 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:49.215333 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:37143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 23:03:49.259097 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 14139
I20260809 23:03:49.276551 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.275953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37143 (local address 127.0.0.1:38540)
0809 23:03:49.276212 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:49.276347 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:37143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 23:03:49.277819 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.277322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37143 (local address 127.0.0.1:38550)
0809 23:03:49.277510 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:49.277643 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:37143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 23:03:49.278167 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 14288
I20260809 23:03:49.285063 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.278943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51470 (local address 127.24.31.195:42693)
0809 23:03:49.279142 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:49.279148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:49.279169 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:03:49.279357 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:49.279363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:49.279462 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:49.279570 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:49.279990 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:49.280729 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:49.282102 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:49.282515 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:49.282651 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:49.283752 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:49.283761 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:49.283775 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:49.284334 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:49.284378 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:49.284397 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:49.284519 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:49.284647 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 23:03:49.284766 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 23:03:49.285717 14435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:49.358413 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.357817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37143 (local address 127.0.0.1:38564)
0809 23:03:49.358074 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:49.358190 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:37143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 23:03:49.359578 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:49.359152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:37143 (local address 127.0.0.1:38572)
0809 23:03:49.359364 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:49.359444 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:37143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 23:03:49.375500 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 14288
I20260809 23:03:49.395635 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 13883
I20260809 23:03:49.396889 13955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:49.508158 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 13883
2026-08-09T23:03:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2831 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:03:49 dist-test-slave-vxj2 krb5kdc[14505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:03:49 dist-test-slave-vxj2 krb5kdc[14505](info): set up 2 sockets
Aug 09 23:03:49 dist-test-slave-vxj2 krb5kdc[14505](info): commencing operation
krb5kdc: starting...
W20260809 23:03:50.717543 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:03:50 dist-test-slave-vxj2 krb5kdc[14505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:03:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:03:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:03:50.993016 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:34571
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:45649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:34571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:51.326313 14519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:51.326623 14519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:51.326788 14519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:51.334049 14519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:03:51.334149 14519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:51.334254 14519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:51.334350 14519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:03:51.334416 14519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:03:51.344194 14519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:45649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:34571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:34571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:51.345492 14519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:51.346845 14519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:51.358425 14525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:51.358561 14529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:51.358525 14527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:51.359683 14519 server_base.cc:1034] running on GCE node
I20260809 23:03:51.360539 14519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:51.362073 14519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:51.363324 14519 hybrid_clock.cc:648] HybridClock initialized: now 1786316631363269 us; error 58 us; skew 500 ppm
I20260809 23:03:51.363677 14519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:51 dist-test-slave-vxj2 krb5kdc[14505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316631, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:51.366942 14519 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:03:51.368445 14519 webserver.cc:460] Webserver started at http://127.24.31.254:45953/ using document root <none> and password file <none>
I20260809 23:03:51.369082 14519 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:51.369257 14519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:51.369535 14519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:51.372418 14519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "5e668168ad2a43879f16061c27ab2175"
format_stamp: "Formatted at 2026-08-09 23:03:51 on dist-test-slave-vxj2"
server_key: "38f2bb008ed8526d0f4c624583bb2f57"
server_key_iv: "ee87853d7a1a25bbb7d7566a99c72c98"
server_key_version: "encryptionkey@0"
I20260809 23:03:51.373354 14519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "5e668168ad2a43879f16061c27ab2175"
format_stamp: "Formatted at 2026-08-09 23:03:51 on dist-test-slave-vxj2"
server_key: "38f2bb008ed8526d0f4c624583bb2f57"
server_key_iv: "ee87853d7a1a25bbb7d7566a99c72c98"
server_key_version: "encryptionkey@0"
I20260809 23:03:51.378545 14519 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 23:03:51.382789 14538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:51.385342 14519 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:03:51.385507 14519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "5e668168ad2a43879f16061c27ab2175"
format_stamp: "Formatted at 2026-08-09 23:03:51 on dist-test-slave-vxj2"
server_key: "38f2bb008ed8526d0f4c624583bb2f57"
server_key_iv: "ee87853d7a1a25bbb7d7566a99c72c98"
server_key_version: "encryptionkey@0"
I20260809 23:03:51.385767 14519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:51.404484 14519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:51.409008 14519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:51.409320 14519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:51.428975 14519 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:34571
I20260809 23:03:51.429008 14605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:34571 every 8 connection(s)
I20260809 23:03:51.430394 14519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:03:51.434353 14606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:51.440202 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 14519
I20260809 23:03:51.440438 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:03:51.440999 24703 external_mini_cluster.cc:1428] Setting key 12d8912aa4f278472566486fa991057d
I20260809 23:03:51.445649 14606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175: Bootstrap starting.
I20260809 23:03:51.449401 14606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:51.450652 14606 log.cc:824] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:51.453622 14606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175: No bootstrap required, opened a new log
Aug 09 23:03:51 dist-test-slave-vxj2 krb5kdc[14505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:51.461843 14606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e668168ad2a43879f16061c27ab2175" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34571 } }
I20260809 23:03:51.462246 14606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:51.462371 14606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e668168ad2a43879f16061c27ab2175, State: Initialized, Role: FOLLOWER
I20260809 23:03:51.463106 14606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e668168ad2a43879f16061c27ab2175" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34571 } }
I20260809 23:03:51.463433 14606 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:03:51.463596 14606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:03:51.463774 14606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:51.464977 14610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:51.443015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.24.31.254:34571)
0809 23:03:51.443888 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:51.443898 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:51.443938 (+ 40us) server_negotiation.cc:407] Connection header received
0809 23:03:51.445755 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:51.445786 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:51.446156 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:51.446454 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:51.448347 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:51.449810 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:51.451684 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:51.452363 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:51.452555 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:51.456908 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:51.456915 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:51.456929 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:51.456971 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:51.460241 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:51.460983 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:51.460993 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:51.460999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:51.461221 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:51.461619 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:51.461625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:51.461628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:51.462247 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:51.462548 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:51.463062 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 23:03:51.463538 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":442,"threads_started":1}
I20260809 23:03:51.467059 14606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e668168ad2a43879f16061c27ab2175" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34571 } }
I20260809 23:03:51.467572 14606 leader_election.cc:302] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e668168ad2a43879f16061c27ab2175; no voters:
I20260809 23:03:51.468302 14606 leader_election.cc:288] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:03:51.468463 14613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:51.469179 14613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [term 1 LEADER]: Becoming Leader. State: Replica: 5e668168ad2a43879f16061c27ab2175, State: Running, Role: LEADER
I20260809 23:03:51.469723 14613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e668168ad2a43879f16061c27ab2175" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34571 } }
I20260809 23:03:51.470295 14606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:03:51.472811 14615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e668168ad2a43879f16061c27ab2175" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e668168ad2a43879f16061c27ab2175" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34571 } } }
I20260809 23:03:51.473253 14615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:51.473618 14616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e668168ad2a43879f16061c27ab2175. Latest consensus state: current_term: 1 leader_uuid: "5e668168ad2a43879f16061c27ab2175" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e668168ad2a43879f16061c27ab2175" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34571 } } }
I20260809 23:03:51.474048 14616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175 [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:51.476648 14622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:03:51.481593 14622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:03:51.489836 14622 catalog_manager.cc:1347] Generated new cluster ID: b98f9e3c69494f69b8da8804a11f144a
I20260809 23:03:51.489953 14622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:03:51.500284 14622 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:03:51.501740 14622 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:03:51.511451 14622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175: Generated new TSK 0
I20260809 23:03:51.512279 14622 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.31.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
Entry for principal kudu/127.24.31.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab.
I20260809 23:03:51.584697 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34571
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:45649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:51.932065 14640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:51.932433 14640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:51.932642 14640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:51.939946 14640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:51.940111 14640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:51.940328 14640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:03:51.950026 14640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:45649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.193.keytab
--principal=kudu/127.24.31.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:51.951382 14640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:51.952914 14640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:51.966898 14646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:51.967155 14650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:51.967375 14645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:51.968143 14640 server_base.cc:1034] running on GCE node
I20260809 23:03:51.968720 14640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:51.969722 14640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:51.970949 14640 hybrid_clock.cc:648] HybridClock initialized: now 1786316631970904 us; error 38 us; skew 500 ppm
I20260809 23:03:51.971319 14640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:51 dist-test-slave-vxj2 krb5kdc[14505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316631, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:51.974726 14640 init.cc:377] Logged in from keytab as kudu/127.24.31.193@KRBTEST.COM (short username kudu)
I20260809 23:03:51.976578 14640 webserver.cc:460] Webserver started at http://127.24.31.193:44153/ using document root <none> and password file <none>
I20260809 23:03:51.977404 14640 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:51.977548 14640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:51.977882 14640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:51.980749 14640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "22ece7afc8574eadbcd6cbf64651eef6"
format_stamp: "Formatted at 2026-08-09 23:03:51 on dist-test-slave-vxj2"
server_key: "6d398721d4b59b339d53e4413e4f0ec3"
server_key_iv: "7f55ca34b70518a51f27cd039a85687b"
server_key_version: "encryptionkey@0"
I20260809 23:03:51.981547 14640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "22ece7afc8574eadbcd6cbf64651eef6"
format_stamp: "Formatted at 2026-08-09 23:03:51 on dist-test-slave-vxj2"
server_key: "6d398721d4b59b339d53e4413e4f0ec3"
server_key_iv: "7f55ca34b70518a51f27cd039a85687b"
server_key_version: "encryptionkey@0"
I20260809 23:03:51.986562 14640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:03:51.990041 14659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:51.991387 14640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:03:51.991549 14640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "22ece7afc8574eadbcd6cbf64651eef6"
format_stamp: "Formatted at 2026-08-09 23:03:51 on dist-test-slave-vxj2"
server_key: "6d398721d4b59b339d53e4413e4f0ec3"
server_key_iv: "7f55ca34b70518a51f27cd039a85687b"
server_key_version: "encryptionkey@0"
I20260809 23:03:51.991748 14640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:52.007758 14640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:52.012161 14640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:52.012475 14640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:52.013505 14640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:52.014829 14640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:52.014955 14640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:52.015102 14640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:52.015180 14640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:52.049763 14640 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:44667
I20260809 23:03:52.049856 14788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:44667 every 8 connection(s)
I20260809 23:03:52.051079 14640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:03:52.057267 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 14640
I20260809 23:03:52.057479 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:03:52.058012 24703 external_mini_cluster.cc:1428] Setting key 4713ad0bfe9fb119b779ce6b146524e9
Aug 09 23:03:52 dist-test-slave-vxj2 krb5kdc[14505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316631, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.193@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:52.073575 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:52.053835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:39547 (local address 127.24.31.254:34571)
0809 23:03:52.054311 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:52.054317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:52.055223 (+ 906us) server_negotiation.cc:407] Connection header received
0809 23:03:52.057074 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:52.057079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:52.057229 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:52.057353 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:52.059092 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:52.059891 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:52.061502 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:52.061909 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:52.062048 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:52.066022 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:52.066033 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:52.066041 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:52.066091 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:52.070039 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:52.070773 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:52.070782 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:52.070788 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:52.070964 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:52.071668 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:52.071675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:52.071678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:52.072017 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:52.072160 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:52.073025 (+ 865us) server_negotiation.cc:299] Negotiation successful
0809 23:03:52.073285 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":253,"threads_started":1}
I20260809 23:03:52.074918 14792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:52.054267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:34571 (local address 127.24.31.193:39547)
0809 23:03:52.055064 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:52.055101 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:52.056815 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:52.057626 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:52.057637 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:52.058234 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:52.058919 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:52.058939 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:52.060070 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:52.060082 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:52.061353 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:52.061365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:52.062149 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:52.062159 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:52.062401 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:52.063143 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:03:52.063181 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:52.065765 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:52.070184 (+ 4419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:52.070191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:52.070202 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:52.070628 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:52.071115 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:52.071122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:52.071126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:52.071416 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:52.072164 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:52.072173 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:52.072636 (+ 463us) client_negotiation.cc:769] Sending connection context
0809 23:03:52.072943 (+ 307us) client_negotiation.cc:240] Negotiation successful
0809 23:03:52.073522 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":330,"threads_started":1}
I20260809 23:03:52.076740 14789 heartbeater.cc:347] Connected to a master server at 127.24.31.254:34571
I20260809 23:03:52.077137 14789 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:52.078037 14789 heartbeater.cc:511] Master 127.24.31.254:34571 requested a full tablet report, sending...
I20260809 23:03:52.080278 14565 ts_manager.cc:194] Registered new tserver with Master: 22ece7afc8574eadbcd6cbf64651eef6 (127.24.31.193:44667)
I20260809 23:03:52.081930 14565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.193@KRBTEST.COM'} at 127.24.31.193:39547
WARNING: no policy specified for kudu/127.24.31.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
Entry for principal kudu/127.24.31.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab.
I20260809 23:03:52.124730 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34571
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:45649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:52.470780 14795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:52.471040 14795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:52.471132 14795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:52.479084 14795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:52.479194 14795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:52.479321 14795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:03:52.488924 14795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:45649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.194.keytab
--principal=kudu/127.24.31.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:52.490334 14795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:52.491703 14795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:52.505590 14800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:52.505525 14801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:52.505820 14803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:52.506855 14795 server_base.cc:1034] running on GCE node
I20260809 23:03:52.507470 14795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:52.508412 14795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:52.509660 14795 hybrid_clock.cc:648] HybridClock initialized: now 1786316632509596 us; error 73 us; skew 500 ppm
I20260809 23:03:52.510025 14795 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:52 dist-test-slave-vxj2 krb5kdc[14505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316632, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:52.513273 14795 init.cc:377] Logged in from keytab as kudu/127.24.31.194@KRBTEST.COM (short username kudu)
I20260809 23:03:52.514750 14795 webserver.cc:460] Webserver started at http://127.24.31.194:37949/ using document root <none> and password file <none>
I20260809 23:03:52.515422 14795 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:52.515527 14795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:52.515793 14795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:52.518666 14795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "4c901c22a5a140a1ba594d71f0f9412d"
format_stamp: "Formatted at 2026-08-09 23:03:52 on dist-test-slave-vxj2"
server_key: "c0c9fb9dbac94d4ad01df23c672a2e37"
server_key_iv: "fbd7cd04bbbb6a545f19d42b8e547f51"
server_key_version: "encryptionkey@0"
I20260809 23:03:52.519409 14795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "4c901c22a5a140a1ba594d71f0f9412d"
format_stamp: "Formatted at 2026-08-09 23:03:52 on dist-test-slave-vxj2"
server_key: "c0c9fb9dbac94d4ad01df23c672a2e37"
server_key_iv: "fbd7cd04bbbb6a545f19d42b8e547f51"
server_key_version: "encryptionkey@0"
I20260809 23:03:52.524418 14795 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 23:03:52.527896 14811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:52.529237 14795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:03:52.529371 14795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "4c901c22a5a140a1ba594d71f0f9412d"
format_stamp: "Formatted at 2026-08-09 23:03:52 on dist-test-slave-vxj2"
server_key: "c0c9fb9dbac94d4ad01df23c672a2e37"
server_key_iv: "fbd7cd04bbbb6a545f19d42b8e547f51"
server_key_version: "encryptionkey@0"
I20260809 23:03:52.529541 14795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:52.542411 14795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:52.546648 14795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:52.546923 14795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:52.547895 14795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:52.549347 14795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:52.549431 14795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:52.549531 14795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:52.549582 14795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:52.584740 14795 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:43407
I20260809 23:03:52.584764 14944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:43407 every 8 connection(s)
I20260809 23:03:52.586248 14795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:03:52.592761 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 14795
I20260809 23:03:52.592980 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:03:52.593489 24703 external_mini_cluster.cc:1428] Setting key eae3d1b790e36760fa37d8164d00041d
Aug 09 23:03:52 dist-test-slave-vxj2 krb5kdc[14505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316632, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.194@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:52.610586 14947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:52.588876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:36405 (local address 127.24.31.254:34571)
0809 23:03:52.589398 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:52.589403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:52.590349 (+ 946us) server_negotiation.cc:407] Connection header received
0809 23:03:52.592606 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:52.592612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:52.592707 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:52.592847 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:52.594757 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:52.595657 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:52.597332 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:52.598118 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:52.598268 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:52.603139 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:52.603148 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:52.603155 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:52.603199 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:52.606781 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:52.607869 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:52.607873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:52.607875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:52.608009 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:52.608536 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:52.608544 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:52.608553 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:52.608868 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:52.609051 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:52.609891 (+ 840us) server_negotiation.cc:299] Negotiation successful
0809 23:03:52.610140 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":289,"threads_started":1}
I20260809 23:03:52.611670 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:52.589416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:34571 (local address 127.24.31.194:36405)
0809 23:03:52.590161 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:52.590205 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:52.592296 (+ 2091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:52.593113 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:52.593132 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:52.593773 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:52.594548 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:52.594577 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:52.595822 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:52.595828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:52.597110 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:52.597130 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:52.598460 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:52.598467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:52.598728 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:52.599528 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:03:52.599562 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:52.602864 (+ 3302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:52.606956 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:52.606968 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:52.606987 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:52.607690 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:52.608131 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:52.608135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:52.608137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:52.608387 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:52.609015 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:52.609024 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:52.609459 (+ 435us) client_negotiation.cc:769] Sending connection context
0809 23:03:52.609774 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 23:03:52.610319 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":307,"threads_started":1}
I20260809 23:03:52.613400 14945 heartbeater.cc:347] Connected to a master server at 127.24.31.254:34571
I20260809 23:03:52.613785 14945 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:52.614588 14945 heartbeater.cc:511] Master 127.24.31.254:34571 requested a full tablet report, sending...
I20260809 23:03:52.616331 14565 ts_manager.cc:194] Registered new tserver with Master: 4c901c22a5a140a1ba594d71f0f9412d (127.24.31.194:43407)
I20260809 23:03:52.617616 14565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.194@KRBTEST.COM'} at 127.24.31.194:36405
WARNING: no policy specified for kudu/127.24.31.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
Entry for principal kudu/127.24.31.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab.
I20260809 23:03:52.658394 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34571
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:45649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:52.989842 14953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:52.990151 14953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:52.990326 14953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:52.997778 14953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:52.997925 14953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:52.998111 14953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:03:53.008215 14953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:45649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.195.keytab
--principal=kudu/127.24.31.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:53.009615 14953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:53.010933 14953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:53.023861 14962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:53.023936 14959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:53.023904 14960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:53.025053 14953 server_base.cc:1034] running on GCE node
I20260809 23:03:53.025686 14953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:53.026604 14953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:53.027838 14953 hybrid_clock.cc:648] HybridClock initialized: now 1786316633027794 us; error 63 us; skew 500 ppm
I20260809 23:03:53.028237 14953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:53 dist-test-slave-vxj2 krb5kdc[14505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316633, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:53.031636 14953 init.cc:377] Logged in from keytab as kudu/127.24.31.195@KRBTEST.COM (short username kudu)
I20260809 23:03:53.033370 14953 webserver.cc:460] Webserver started at http://127.24.31.195:46309/ using document root <none> and password file <none>
I20260809 23:03:53.034145 14953 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:53.034303 14953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:53.034650 14953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:53.037869 14953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "cc1afabcc2bb4045ac1cdba47ca26175"
format_stamp: "Formatted at 2026-08-09 23:03:53 on dist-test-slave-vxj2"
server_key: "e2e783ee3d8c97cddc9ceb213c73c60d"
server_key_iv: "62ed4c9ce3c561ad825c4c7d1d3f4445"
server_key_version: "encryptionkey@0"
I20260809 23:03:53.038713 14953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "cc1afabcc2bb4045ac1cdba47ca26175"
format_stamp: "Formatted at 2026-08-09 23:03:53 on dist-test-slave-vxj2"
server_key: "e2e783ee3d8c97cddc9ceb213c73c60d"
server_key_iv: "62ed4c9ce3c561ad825c4c7d1d3f4445"
server_key_version: "encryptionkey@0"
I20260809 23:03:53.044169 14953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 23:03:53.047748 14970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:53.049100 14953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:03:53.049348 14953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "cc1afabcc2bb4045ac1cdba47ca26175"
format_stamp: "Formatted at 2026-08-09 23:03:53 on dist-test-slave-vxj2"
server_key: "e2e783ee3d8c97cddc9ceb213c73c60d"
server_key_iv: "62ed4c9ce3c561ad825c4c7d1d3f4445"
server_key_version: "encryptionkey@0"
I20260809 23:03:53.049562 14953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:53.067205 14953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:53.071357 14953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:53.071664 14953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:53.072714 14953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:53.074167 14953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:53.074276 14953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:53.074436 14953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:53.074512 14953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:53.084966 14789 heartbeater.cc:503] Master 127.24.31.254:34571 was elected leader, sending a full tablet report...
I20260809 23:03:53.109540 14953 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:40473
I20260809 23:03:53.109586 15096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:40473 every 8 connection(s)
I20260809 23:03:53.110855 14953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:03:53.113085 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 14953
I20260809 23:03:53.113322 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:03:53.113932 24703 external_mini_cluster.cc:1428] Setting key c8cda9c417a6bde7f6b6c10b1659ec27
Aug 09 23:03:53 dist-test-slave-vxj2 krb5kdc[14505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316633, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.195@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:03:53.131116 15099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.113549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:37875 (local address 127.24.31.254:34571)
0809 23:03:53.114058 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.114065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.115078 (+ 1013us) server_negotiation.cc:407] Connection header received
0809 23:03:53.117440 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.117445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.117533 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.117642 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:53.119104 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.119917 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.121520 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.121956 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.122093 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.125375 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:53.125380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:53.125383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:53.125413 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:53.128184 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.129091 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.129096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.129098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.129289 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.129699 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.129703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.129704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.129956 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:53.130076 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.130663 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.130834 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":238,"threads_started":1}
I20260809 23:03:53.131994 15100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.114093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:34571 (local address 127.24.31.195:37875)
0809 23:03:53.114927 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:53.114971 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:53.117128 (+ 2157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:53.117884 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:53.117898 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:53.118380 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:53.118936 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.118953 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.120077 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.120083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.121377 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.121390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.122176 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.122185 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.122435 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.123041 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:03:53.123062 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:53.125135 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:53.128350 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:53.128372 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:53.128388 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:53.128969 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:53.129400 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:03:53.129403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:03:53.129406 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:03:53.129589 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:03:53.130061 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:53.130066 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:53.130343 (+ 277us) client_negotiation.cc:769] Sending connection context
0809 23:03:53.130569 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 23:03:53.130903 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":303,"threads_started":1}
I20260809 23:03:53.133333 15097 heartbeater.cc:347] Connected to a master server at 127.24.31.254:34571
I20260809 23:03:53.133579 15097 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:53.134255 15097 heartbeater.cc:511] Master 127.24.31.254:34571 requested a full tablet report, sending...
I20260809 23:03:53.135691 14565 ts_manager.cc:194] Registered new tserver with Master: cc1afabcc2bb4045ac1cdba47ca26175 (127.24.31.195:40473)
I20260809 23:03:53.136514 14565 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.31.195@KRBTEST.COM'} at 127.24.31.195:37875
I20260809 23:03:53.141275 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:03:53.158793 15099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.145441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54456 (local address 127.24.31.254:34571)
0809 23:03:53.145601 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.145605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.146032 (+ 427us) server_negotiation.cc:407] Connection header received
0809 23:03:53.146169 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.146172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.146243 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.146332 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:53.147749 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.148505 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.149876 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.150295 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.150406 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.152959 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:53.152965 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:53.152968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:53.153018 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:53.156040 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.156753 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.156760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.156764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.156906 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.157344 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.157350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.157357 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.157658 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:53.157935 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.158272 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.158486 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:03:53.162323 14565 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:03:53.169776 14565 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:03:53.206099 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.196910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34138 (local address 127.24.31.195:40473)
0809 23:03:53.198876 (+ 1966us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.198883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.198914 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:03:53.199003 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.199011 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.199206 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.199408 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:53.200004 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.201484 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.203773 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.204906 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.205052 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.205305 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.205576 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.205731 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1780,"thread_start_us":1617,"threads_started":1}
I20260809 23:03:53.206158 15118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.196744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:40473 (local address 127.0.0.1:34138)
0809 23:03:53.198159 (+ 1415us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:53.198181 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:53.198288 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:53.199432 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:53.199437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:53.199463 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:53.199812 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.199823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.201648 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.201653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.203535 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.203550 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.205101 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.205108 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.205337 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.205384 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 23:03:53.205505 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 23:03:53.205721 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1286,"mutex_wait_us":105,"thread_start_us":1074,"threads_started":1}
I20260809 23:03:53.206734 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.182753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:44667 (local address 127.0.0.1:41276)
0809 23:03:53.195720 (+ 12967us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:53.195744 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:53.195948 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:53.197629 (+ 1681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:53.197640 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:53.197691 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:53.198304 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.198322 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.200488 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.200496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.203420 (+ 2924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.203432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.205747 (+ 2315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.205753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.205952 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.206001 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 23:03:53.206091 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 23:03:53.206233 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12827,"mutex_wait_us":201,"spinlock_wait_cycles":32512,"thread_start_us":13353,"threads_started":1}
I20260809 23:03:53.207484 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.183210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.24.31.193:44667)
0809 23:03:53.196594 (+ 13384us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.196613 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.196640 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:03:53.196743 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.196751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.197015 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.197241 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:53.198641 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.200343 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.203686 (+ 3343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.205228 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.205371 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.205594 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.206981 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.207167 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13194,"thread_start_us":13106,"threads_started":1}
I20260809 23:03:53.207497 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.182703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.24.31.194:43407)
0809 23:03:53.195684 (+ 12981us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.195695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.196575 (+ 880us) server_negotiation.cc:407] Connection header received
0809 23:03:53.196710 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.196718 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.197042 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.197301 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:53.199037 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.200436 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.203800 (+ 3364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.205230 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.205367 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.205569 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.207054 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.207210 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12454,"thread_start_us":12351,"threads_started":1}
I20260809 23:03:53.208187 15116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.182521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:43407 (local address 127.0.0.1:36650)
0809 23:03:53.195702 (+ 13181us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:53.195740 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:53.195946 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:53.198075 (+ 2129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:53.198080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:53.198103 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:53.198873 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.198883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.200607 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.200612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.203548 (+ 2936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.203557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.206538 (+ 2981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.206547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.206746 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.206788 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 23:03:53.207692 (+ 904us) client_negotiation.cc:240] Negotiation successful
0809 23:03:53.207823 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12906,"mutex_wait_us":161,"spinlock_wait_cycles":3456,"thread_start_us":12914,"threads_started":1}
I20260809 23:03:53.211000 14710 tablet_service.cc:1467] Processing CreateTablet for tablet fc315ce30a79443caa92e30649da778e (DEFAULT_TABLE table=test-table [id=4bd7b162683b497e983340eea32a29f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:53.212631 14710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc315ce30a79443caa92e30649da778e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:53.212858 14869 tablet_service.cc:1467] Processing CreateTablet for tablet fc315ce30a79443caa92e30649da778e (DEFAULT_TABLE table=test-table [id=4bd7b162683b497e983340eea32a29f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:53.210352 15025 tablet_service.cc:1467] Processing CreateTablet for tablet fc315ce30a79443caa92e30649da778e (DEFAULT_TABLE table=test-table [id=4bd7b162683b497e983340eea32a29f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:53.214210 14869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc315ce30a79443caa92e30649da778e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:53.214460 15025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc315ce30a79443caa92e30649da778e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:53.225244 15120 tablet_bootstrap.cc:492] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: Bootstrap starting.
I20260809 23:03:53.227360 15123 tablet_bootstrap.cc:492] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: Bootstrap starting.
I20260809 23:03:53.228875 15124 tablet_bootstrap.cc:492] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: Bootstrap starting.
I20260809 23:03:53.230123 15120 tablet_bootstrap.cc:654] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:53.230850 15123 tablet_bootstrap.cc:654] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:53.231446 15120 log.cc:824] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:53.231585 15124 tablet_bootstrap.cc:654] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:53.232111 15123 log.cc:824] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:53.232795 15124 log.cc:824] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:53.234658 15120 tablet_bootstrap.cc:492] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: No bootstrap required, opened a new log
I20260809 23:03:53.234934 15120 ts_tablet_manager.cc:1397] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 23:03:53.235356 15124 tablet_bootstrap.cc:492] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: No bootstrap required, opened a new log
I20260809 23:03:53.235605 15124 ts_tablet_manager.cc:1397] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 23:03:53.242041 15120 raft_consensus.cc:348] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } }
I20260809 23:03:53.242419 15120 raft_consensus.cc:374] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:53.242583 15120 raft_consensus.cc:729] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1afabcc2bb4045ac1cdba47ca26175, State: Initialized, Role: FOLLOWER
I20260809 23:03:53.242499 15124 raft_consensus.cc:348] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } }
I20260809 23:03:53.242951 15124 raft_consensus.cc:374] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:53.243064 15124 raft_consensus.cc:729] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c901c22a5a140a1ba594d71f0f9412d, State: Initialized, Role: FOLLOWER
I20260809 23:03:53.243316 15120 consensus_queue.cc:260] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } }
I20260809 23:03:53.243649 15124 consensus_queue.cc:260] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } }
I20260809 23:03:53.244714 15097 heartbeater.cc:503] Master 127.24.31.254:34571 was elected leader, sending a full tablet report...
I20260809 23:03:53.244980 15123 tablet_bootstrap.cc:492] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: No bootstrap required, opened a new log
I20260809 23:03:53.245275 15120 ts_tablet_manager.cc:1428] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: Time spent starting tablet: real 0.010s user 0.007s sys 0.005s
I20260809 23:03:53.245306 15123 ts_tablet_manager.cc:1397] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.006s
I20260809 23:03:53.246224 14945 heartbeater.cc:503] Master 127.24.31.254:34571 was elected leader, sending a full tablet report...
I20260809 23:03:53.246887 15124 ts_tablet_manager.cc:1428] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260809 23:03:53.254537 15123 raft_consensus.cc:348] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } }
I20260809 23:03:53.254910 15123 raft_consensus.cc:374] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:53.255015 15123 raft_consensus.cc:729] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ece7afc8574eadbcd6cbf64651eef6, State: Initialized, Role: FOLLOWER
I20260809 23:03:53.255527 15123 consensus_queue.cc:260] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } }
I20260809 23:03:53.256858 15123 ts_tablet_manager.cc:1428] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20260809 23:03:53.303588 14790 tablet.cc:2369] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:53.326345 15128 raft_consensus.cc:482] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:03:53.326870 15128 raft_consensus.cc:504] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } }
I20260809 23:03:53.328656 15128 leader_election.cc:288] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c901c22a5a140a1ba594d71f0f9412d (127.24.31.194:43407), 22ece7afc8574eadbcd6cbf64651eef6 (127.24.31.193:44667)
I20260809 23:03:53.334481 15100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.329021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:44667 (local address 127.24.31.195:44625)
0809 23:03:53.329220 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:53.329241 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:53.329404 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:53.329875 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:53.329887 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:53.329916 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:53.330094 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.330102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.331128 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.331135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.332411 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.332452 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.333728 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.333734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.333940 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.333964 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 23:03:53.334066 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 23:03:53.334196 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":19}
I20260809 23:03:53.334481 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.329275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:44625 (local address 127.24.31.193:44667)
0809 23:03:53.329492 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.329498 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.329520 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:03:53.329608 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.329615 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.329708 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.329827 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:53.330248 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.330978 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.332639 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.333558 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.333733 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.333820 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.334105 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.334229 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 23:03:53.335454 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.329289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:49473 (local address 127.24.31.194:43407)
0809 23:03:53.329491 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.329498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.330209 (+ 711us) server_negotiation.cc:407] Connection header received
0809 23:03:53.330410 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.330414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.330473 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.330575 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:53.331099 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.331772 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.333568 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.334480 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.334675 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.334773 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.335179 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.335276 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 23:03:53.335647 15131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.329035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:43407 (local address 127.24.31.195:49473)
0809 23:03:53.330092 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:53.330106 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:53.330256 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:53.330586 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:53.330590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:53.330612 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:53.330935 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.330944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.331926 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.331932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.333406 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:53.333418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.334796 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.334802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:53.334964 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.335004 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:03:53.335138 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 23:03:53.335277 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"mutex_wait_us":160,"thread_start_us":463,"threads_started":1}
I20260809 23:03:53.335626 14732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc315ce30a79443caa92e30649da778e" candidate_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ece7afc8574eadbcd6cbf64651eef6" is_pre_election: true
I20260809 23:03:53.336194 14732 raft_consensus.cc:2454] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1afabcc2bb4045ac1cdba47ca26175 in term 0.
I20260809 23:03:53.336621 14893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc315ce30a79443caa92e30649da778e" candidate_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c901c22a5a140a1ba594d71f0f9412d" is_pre_election: true
I20260809 23:03:53.336951 14972 leader_election.cc:302] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ece7afc8574eadbcd6cbf64651eef6, cc1afabcc2bb4045ac1cdba47ca26175; no voters:
I20260809 23:03:53.337132 14893 raft_consensus.cc:2454] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1afabcc2bb4045ac1cdba47ca26175 in term 0.
I20260809 23:03:53.337360 15128 raft_consensus.cc:2781] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:03:53.337538 15128 raft_consensus.cc:482] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:03:53.337666 15128 raft_consensus.cc:3037] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 0 FOLLOWER]: Advancing to term 1
W20260809 23:03:53.338333 14946 tablet.cc:2369] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:53.340359 15128 raft_consensus.cc:504] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } }
I20260809 23:03:53.340911 15128 leader_election.cc:288] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [CANDIDATE]: Term 1 election: Requested vote from peers 4c901c22a5a140a1ba594d71f0f9412d (127.24.31.194:43407), 22ece7afc8574eadbcd6cbf64651eef6 (127.24.31.193:44667)
I20260809 23:03:53.341387 14893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc315ce30a79443caa92e30649da778e" candidate_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c901c22a5a140a1ba594d71f0f9412d"
I20260809 23:03:53.341423 14732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc315ce30a79443caa92e30649da778e" candidate_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ece7afc8574eadbcd6cbf64651eef6"
I20260809 23:03:53.341603 14893 raft_consensus.cc:3037] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:53.341609 14732 raft_consensus.cc:3037] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:53.344071 14732 raft_consensus.cc:2454] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1afabcc2bb4045ac1cdba47ca26175 in term 1.
I20260809 23:03:53.344305 14893 raft_consensus.cc:2454] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1afabcc2bb4045ac1cdba47ca26175 in term 1.
I20260809 23:03:53.344486 14972 leader_election.cc:302] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ece7afc8574eadbcd6cbf64651eef6, cc1afabcc2bb4045ac1cdba47ca26175; no voters:
I20260809 23:03:53.344771 15128 raft_consensus.cc:2781] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:53.345286 15128 raft_consensus.cc:686] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 1 LEADER]: Becoming Leader. State: Replica: cc1afabcc2bb4045ac1cdba47ca26175, State: Running, Role: LEADER
I20260809 23:03:53.345726 15128 consensus_queue.cc:237] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } }
I20260809 23:03:53.350083 14565 catalog_manager.cc:5495] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc1afabcc2bb4045ac1cdba47ca26175 (127.24.31.195). New cstate: current_term: 1 leader_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1afabcc2bb4045ac1cdba47ca26175" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 40473 } health_report { overall_health: HEALTHY } } }
W20260809 23:03:53.362447 15098 tablet.cc:2369] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:53.402216 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.396348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.24.31.195:40473)
0809 23:03:53.396580 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.396587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.396754 (+ 167us) server_negotiation.cc:407] Connection header received
0809 23:03:53.397014 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.397018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.397104 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.397329 (+ 225us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:03:53.397772 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.398589 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.400034 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.400446 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.400554 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.401036 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:03:53.401598 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:03:53.401717 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.401855 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.401924 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 23:03:53.410874 14732 raft_consensus.cc:1261] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [term 1 FOLLOWER]: Refusing update from remote peer cc1afabcc2bb4045ac1cdba47ca26175: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:53.410883 14893 raft_consensus.cc:1261] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [term 1 FOLLOWER]: Refusing update from remote peer cc1afabcc2bb4045ac1cdba47ca26175: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:53.411892 15132 consensus_queue.cc:1035] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ece7afc8574eadbcd6cbf64651eef6" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 44667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:53.412248 15128 consensus_queue.cc:1035] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c901c22a5a140a1ba594d71f0f9412d" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 43407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:53.424012 15135 mvcc.cc:205] Tried to move back new op lower bound from 7316752930440306688 to 7316752930188042240. Current Snapshot: MvccSnapshot[applied={T|T < 7316752930440306688}]
I20260809 23:03:53.440797 14563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:53.441118 14563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:53.445803 14563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e668168ad2a43879f16061c27ab2175: Sending DeleteTablet for 3 replicas of tablet fc315ce30a79443caa92e30649da778e
I20260809 23:03:53.447103 14710 tablet_service.cc:1514] Processing DeleteTablet for tablet fc315ce30a79443caa92e30649da778e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:53 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:41276
I20260809 23:03:53.447544 15025 tablet_service.cc:1514] Processing DeleteTablet for tablet fc315ce30a79443caa92e30649da778e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:53 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:34138
I20260809 23:03:53.447890 14869 tablet_service.cc:1514] Processing DeleteTablet for tablet fc315ce30a79443caa92e30649da778e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:53 UTC) from {username='kudu', principal='kudu/127.24.31.254@KRBTEST.COM'} at 127.0.0.1:36650
I20260809 23:03:53.449911 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 14640
I20260809 23:03:53.458808 15146 tablet_replica.cc:330] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: stopping tablet replica
I20260809 23:03:53.459524 15146 raft_consensus.cc:2229] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:53.460213 15146 raft_consensus.cc:2258] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:53.460832 15147 tablet_replica.cc:330] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: stopping tablet replica
I20260809 23:03:53.461220 15147 raft_consensus.cc:2229] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:03:53.461805 15147 raft_consensus.cc:2258] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:53.463816 15148 tablet_replica.cc:330] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: stopping tablet replica
I20260809 23:03:53.464073 15147 ts_tablet_manager.cc:1905] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:53.464072 15146 ts_tablet_manager.cc:1905] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:53.464238 15148 raft_consensus.cc:2229] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:53.464735 15148 raft_consensus.cc:2258] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:53.467124 15148 ts_tablet_manager.cc:1905] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:53.471127 15147 ts_tablet_manager.cc:1918] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:53.471385 15147 log.cc:1196] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/fc315ce30a79443caa92e30649da778e
I20260809 23:03:53.471536 15146 ts_tablet_manager.cc:1918] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:53.471707 15146 log.cc:1196] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/fc315ce30a79443caa92e30649da778e
I20260809 23:03:53.472172 15147 ts_tablet_manager.cc:1939] T fc315ce30a79443caa92e30649da778e P cc1afabcc2bb4045ac1cdba47ca26175: Deleting consensus metadata
I20260809 23:03:53.472309 15146 ts_tablet_manager.cc:1939] T fc315ce30a79443caa92e30649da778e P 22ece7afc8574eadbcd6cbf64651eef6: Deleting consensus metadata
I20260809 23:03:53.474138 14541 catalog_manager.cc:4841] TS 22ece7afc8574eadbcd6cbf64651eef6 (127.24.31.193:44667): tablet fc315ce30a79443caa92e30649da778e (table test-table [id=4bd7b162683b497e983340eea32a29f5]) successfully deleted
I20260809 23:03:53.474519 14541 catalog_manager.cc:4841] TS cc1afabcc2bb4045ac1cdba47ca26175 (127.24.31.195:40473): tablet fc315ce30a79443caa92e30649da778e (table test-table [id=4bd7b162683b497e983340eea32a29f5]) successfully deleted
I20260809 23:03:53.474609 15148 ts_tablet_manager.cc:1918] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:53.474794 15148 log.cc:1196] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/fc315ce30a79443caa92e30649da778e
I20260809 23:03:53.475421 15148 ts_tablet_manager.cc:1939] T fc315ce30a79443caa92e30649da778e P 4c901c22a5a140a1ba594d71f0f9412d: Deleting consensus metadata
Aug 09 23:03:53 dist-test-slave-vxj2 krb5kdc[14505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.193@KRBTEST.COM
I20260809 23:03:53.477311 14540 catalog_manager.cc:4841] TS 4c901c22a5a140a1ba594d71f0f9412d (127.24.31.194:43407): tablet fc315ce30a79443caa92e30649da778e (table test-table [id=4bd7b162683b497e983340eea32a29f5]) successfully deleted
I20260809 23:03:53.484757 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.451542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41284 (local address 127.24.31.193:44667)
0809 23:03:53.451771 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.451780 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.453777 (+ 1997us) server_negotiation.cc:407] Connection header received
0809 23:03:53.465540 (+ 11763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.465547 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.465668 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.465786 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:53.467964 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.469200 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.471385 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.472125 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.472261 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.478262 (+ 6001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:53.478276 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:53.478290 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:53.478326 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:53.481566 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.482385 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.482392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.482395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.482534 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.483007 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.483010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.483012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.483401 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:53.483743 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.484117 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.484358 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 23:03:53.485483 14782 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:53.579881 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 14640
I20260809 23:03:53.606889 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 14795
Aug 09 23:03:53 dist-test-slave-vxj2 krb5kdc[14505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.194@KRBTEST.COM
I20260809 23:03:53.628683 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.608019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36656 (local address 127.24.31.194:43407)
0809 23:03:53.608207 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.608214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.608235 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:03:53.608299 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.608305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.608401 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.608523 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:53.610555 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.611821 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.614055 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.614755 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.614886 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.621292 (+ 6406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:53.621310 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:53.621328 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:53.621383 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:53.625103 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.626140 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.626149 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.626151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.626286 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.626966 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.626970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.626972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.627388 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:53.627727 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.628066 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.628300 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 23:03:53.629670 14940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:53.750447 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 14795
I20260809 23:03:53.771154 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 14953
Aug 09 23:03:53 dist-test-slave-vxj2 krb5kdc[14505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.195@KRBTEST.COM
I20260809 23:03:53.789880 15119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:53.772073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.24.31.195:40473)
0809 23:03:53.772577 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:53.772581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:53.772599 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:03:53.772662 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:53.772666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:53.772744 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:53.772879 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:53.774519 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.775425 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.777014 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:53.777586 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:53.777781 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:53.782663 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:53.782679 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:53.782700 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:53.782750 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:53.786349 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.787211 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.787223 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.787225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.787413 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:53.787888 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:53.787895 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:53.787899 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:53.788322 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:53.788660 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:53.789194 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 23:03:53.789470 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376}
I20260809 23:03:53.790935 15092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:53.918718 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 14953
I20260809 23:03:53.941308 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 14519
I20260809 23:03:53.942548 14601 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:54.055403 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 14519
2026-08-09T23:03:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4546 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T23:03:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:03:54Z Disabled control of system clock
I20260809 23:03:54.141253 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:34047
--webserver_interface=127.24.31.254
--webserver_port=0
--builtin_ntp_servers=127.24.31.212:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:34047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:03:54.477365 15158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:54.477665 15158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:54.477749 15158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:54.485069 15158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:03:54.485225 15158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:54.485273 15158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:54.485337 15158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:03:54.485383 15158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:03:54.494411 15158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:34047
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:34047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:54.496037 15158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:54.497879 15158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:54.510008 15169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:54.510044 15167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:54.510085 15166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:54.511116 15158 server_base.cc:1034] running on GCE node
I20260809 23:03:54.511900 15158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:54.513266 15158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:54.514494 15158 hybrid_clock.cc:648] HybridClock initialized: now 1786316634514437 us; error 66 us; skew 500 ppm
I20260809 23:03:54.514844 15158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:54.520253 15158 webserver.cc:460] Webserver started at http://127.24.31.254:36543/ using document root <none> and password file <none>
I20260809 23:03:54.520934 15158 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:54.521071 15158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:54.521430 15158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:54.524345 15158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "9ccf8843edfd40379b2ab566a88551bc"
format_stamp: "Formatted at 2026-08-09 23:03:54 on dist-test-slave-vxj2"
server_key: "5ee77f3e19d299600e0f29ae730f265a"
server_key_iv: "d26663de1a04d43965661c7f69a4b57b"
server_key_version: "encryptionkey@0"
I20260809 23:03:54.525187 15158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "9ccf8843edfd40379b2ab566a88551bc"
format_stamp: "Formatted at 2026-08-09 23:03:54 on dist-test-slave-vxj2"
server_key: "5ee77f3e19d299600e0f29ae730f265a"
server_key_iv: "d26663de1a04d43965661c7f69a4b57b"
server_key_version: "encryptionkey@0"
I20260809 23:03:54.530342 15158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:03:54.534066 15176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:54.535457 15158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:03:54.535666 15158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "9ccf8843edfd40379b2ab566a88551bc"
format_stamp: "Formatted at 2026-08-09 23:03:54 on dist-test-slave-vxj2"
server_key: "5ee77f3e19d299600e0f29ae730f265a"
server_key_iv: "d26663de1a04d43965661c7f69a4b57b"
server_key_version: "encryptionkey@0"
I20260809 23:03:54.535959 15158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:54.567122 15158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:54.568089 15158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:54.568457 15158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:54.588902 15158 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:34047
I20260809 23:03:54.588923 15237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:34047 every 8 connection(s)
I20260809 23:03:54.590411 15158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:03:54.594231 15238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:54.595290 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 15158
I20260809 23:03:54.595477 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:03:54.596065 24703 external_mini_cluster.cc:1428] Setting key 74cd551433f8b34a2425038459250c70
I20260809 23:03:54.608759 15238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc: Bootstrap starting.
I20260809 23:03:54.611364 15241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:54.598168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.24.31.254:34047)
0809 23:03:54.599230 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:54.599240 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:54.599301 (+ 61us) server_negotiation.cc:407] Connection header received
0809 23:03:54.601109 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:54.601131 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:54.601669 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:54.602016 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:54.602626 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:54.604485 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:54.606012 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:54.607079 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:54.607273 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:54.608460 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:54.608470 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:54.608486 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:54.609197 (+ 711us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:54.609306 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:54.609336 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:54.609450 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:54.609714 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 23:03:54.610019 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":418,"threads_started":1}
I20260809 23:03:54.612797 15238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:54.613948 15238 log.cc:824] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:54.616793 15238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc: No bootstrap required, opened a new log
I20260809 23:03:54.624004 15238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccf8843edfd40379b2ab566a88551bc" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34047 } }
I20260809 23:03:54.624358 15238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:54.624468 15238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ccf8843edfd40379b2ab566a88551bc, State: Initialized, Role: FOLLOWER
I20260809 23:03:54.624981 15238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccf8843edfd40379b2ab566a88551bc" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34047 } }
I20260809 23:03:54.625274 15238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:03:54.625413 15238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:03:54.625599 15238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:54.627851 15238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccf8843edfd40379b2ab566a88551bc" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34047 } }
I20260809 23:03:54.628271 15238 leader_election.cc:302] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ccf8843edfd40379b2ab566a88551bc; no voters:
I20260809 23:03:54.628818 15238 leader_election.cc:288] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:03:54.628994 15243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:54.629878 15243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [term 1 LEADER]: Becoming Leader. State: Replica: 9ccf8843edfd40379b2ab566a88551bc, State: Running, Role: LEADER
I20260809 23:03:54.630419 15243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccf8843edfd40379b2ab566a88551bc" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34047 } }
I20260809 23:03:54.631004 15238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:03:54.633037 15245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ccf8843edfd40379b2ab566a88551bc. Latest consensus state: current_term: 1 leader_uuid: "9ccf8843edfd40379b2ab566a88551bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccf8843edfd40379b2ab566a88551bc" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34047 } } }
I20260809 23:03:54.633486 15245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:54.635999 15249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:03:54.635963 15244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ccf8843edfd40379b2ab566a88551bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ccf8843edfd40379b2ab566a88551bc" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 34047 } } }
I20260809 23:03:54.636513 15244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:54.641664 15249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:03:54.651481 15249 catalog_manager.cc:1347] Generated new cluster ID: e30abf0e044f484eae995b8ca55725d4
I20260809 23:03:54.651602 15249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:03:54.668633 15249 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:03:54.670498 15249 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:03:54.689070 15249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc: Generated new TSK 0
I20260809 23:03:54.690291 15249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:03:54.696616 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.193:0
--local_ip_for_outbound_sockets=127.24.31.193
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34047
--builtin_ntp_servers=127.24.31.212:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:03:55.033998 15266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:55.034242 15266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:55.034409 15266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:55.042101 15266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:55.042193 15266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:55.042366 15266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.193
I20260809 23:03:55.051358 15266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.31.193
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.193
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:55.052552 15266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:55.054369 15266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:55.067798 15275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:55.067834 15273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:55.067876 15272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:55.068811 15266 server_base.cc:1034] running on GCE node
I20260809 23:03:55.069499 15266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:55.070410 15266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:55.071646 15266 hybrid_clock.cc:648] HybridClock initialized: now 1786316635071604 us; error 38 us; skew 500 ppm
I20260809 23:03:55.072057 15266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:55.074369 15266 webserver.cc:460] Webserver started at http://127.24.31.193:36671/ using document root <none> and password file <none>
I20260809 23:03:55.074990 15266 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:55.075084 15266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:55.075340 15266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:55.078171 15266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/data/instance:
uuid: "c7b110ac06e74e9bbced530d42d5d0ad"
format_stamp: "Formatted at 2026-08-09 23:03:55 on dist-test-slave-vxj2"
server_key: "8871dc9434fbb3b08360449e7132cb7e"
server_key_iv: "f6d35d06f8fd605524f00ee49a28370b"
server_key_version: "encryptionkey@0"
I20260809 23:03:55.078896 15266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance:
uuid: "c7b110ac06e74e9bbced530d42d5d0ad"
format_stamp: "Formatted at 2026-08-09 23:03:55 on dist-test-slave-vxj2"
server_key: "8871dc9434fbb3b08360449e7132cb7e"
server_key_iv: "f6d35d06f8fd605524f00ee49a28370b"
server_key_version: "encryptionkey@0"
I20260809 23:03:55.083810 15266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:03:55.087409 15282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:55.088701 15266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 23:03:55.088889 15266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/wal
uuid: "c7b110ac06e74e9bbced530d42d5d0ad"
format_stamp: "Formatted at 2026-08-09 23:03:55 on dist-test-slave-vxj2"
server_key: "8871dc9434fbb3b08360449e7132cb7e"
server_key_iv: "f6d35d06f8fd605524f00ee49a28370b"
server_key_version: "encryptionkey@0"
I20260809 23:03:55.089073 15266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:55.105639 15266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:55.106397 15266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:55.106712 15266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:55.107719 15266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:55.109360 15266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:55.109472 15266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:55.109596 15266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:55.109714 15266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:55.144855 15266 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.193:34483
I20260809 23:03:55.144943 15410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.193:34483 every 8 connection(s)
I20260809 23:03:55.146214 15266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/data/info.pb
I20260809 23:03:55.147733 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 15266
I20260809 23:03:55.147928 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/wal/instance
I20260809 23:03:55.148404 24703 external_mini_cluster.cc:1428] Setting key a25bf6be1ed1999aa94a6eb45b18e154
I20260809 23:03:55.152257 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.194:0
--local_ip_for_outbound_sockets=127.24.31.194
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34047
--builtin_ntp_servers=127.24.31.212:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:03:55.162705 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:55.149067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.193:37801 (local address 127.24.31.254:34047)
0809 23:03:55.152975 (+ 3908us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:55.152979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:55.153846 (+ 867us) server_negotiation.cc:407] Connection header received
0809 23:03:55.155753 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:55.155759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:55.155851 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:55.155980 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:55.157093 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:55.157927 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:55.159625 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:55.160065 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:55.160214 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:55.161764 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:55.161770 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:55.161774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:55.161795 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:55.161843 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:55.161858 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:55.161971 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:55.162205 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 23:03:55.162315 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3803,"thread_start_us":3685,"threads_started":1}
I20260809 23:03:55.163695 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:55.149783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:34047 (local address 127.24.31.193:37801)
0809 23:03:55.153688 (+ 3905us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:55.153729 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:55.155470 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:55.156238 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:55.156247 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:55.156332 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:55.156914 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:55.156931 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:55.158107 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:55.158113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:55.159464 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:55.159479 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:55.160332 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:55.160340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:55.160596 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:55.161378 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:03:55.161381 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:55.161420 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:03:55.161496 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:55.161971 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:55.161986 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:55.161998 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 23:03:55.162193 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 23:03:55.162593 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3466,"thread_start_us":3216,"threads_started":1}
I20260809 23:03:55.165374 15412 heartbeater.cc:347] Connected to a master server at 127.24.31.254:34047
I20260809 23:03:55.165668 15412 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:55.166436 15412 heartbeater.cc:511] Master 127.24.31.254:34047 requested a full tablet report, sending...
I20260809 23:03:55.168660 15195 ts_manager.cc:194] Registered new tserver with Master: c7b110ac06e74e9bbced530d42d5d0ad (127.24.31.193:34483)
I20260809 23:03:55.170938 15195 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.193:37801
W20260809 23:03:55.494809 15417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:55.495080 15417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:55.495165 15417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:55.502378 15417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:55.502473 15417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:55.502588 15417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.194
I20260809 23:03:55.511855 15417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.31.194
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.194
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:55.513075 15417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:55.514787 15417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:55.527568 15424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:55.527547 15427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:55.527966 15425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:55.530674 15417 server_base.cc:1034] running on GCE node
I20260809 23:03:55.531337 15417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:55.532387 15417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:55.533609 15417 hybrid_clock.cc:648] HybridClock initialized: now 1786316635533562 us; error 39 us; skew 500 ppm
I20260809 23:03:55.533959 15417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:55.536212 15417 webserver.cc:460] Webserver started at http://127.24.31.194:45987/ using document root <none> and password file <none>
I20260809 23:03:55.536928 15417 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:55.537056 15417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:55.537401 15417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:55.540287 15417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/data/instance:
uuid: "ca11f079f4114a2389459b33aff8b081"
format_stamp: "Formatted at 2026-08-09 23:03:55 on dist-test-slave-vxj2"
server_key: "9517f5827884def5f0b969673c5fe38e"
server_key_iv: "9d4494b79fc862a7be2daf4a1f03b66a"
server_key_version: "encryptionkey@0"
I20260809 23:03:55.541114 15417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance:
uuid: "ca11f079f4114a2389459b33aff8b081"
format_stamp: "Formatted at 2026-08-09 23:03:55 on dist-test-slave-vxj2"
server_key: "9517f5827884def5f0b969673c5fe38e"
server_key_iv: "9d4494b79fc862a7be2daf4a1f03b66a"
server_key_version: "encryptionkey@0"
I20260809 23:03:55.546427 15417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 23:03:55.551720 15433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:55.553779 15417 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 23:03:55.553982 15417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/wal
uuid: "ca11f079f4114a2389459b33aff8b081"
format_stamp: "Formatted at 2026-08-09 23:03:55 on dist-test-slave-vxj2"
server_key: "9517f5827884def5f0b969673c5fe38e"
server_key_iv: "9d4494b79fc862a7be2daf4a1f03b66a"
server_key_version: "encryptionkey@0"
I20260809 23:03:55.554272 15417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:55.592320 15417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:55.593461 15417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:55.593818 15417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:55.594933 15417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:55.596468 15417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:55.596562 15417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:55.596737 15417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:55.596843 15417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:55.633746 15417 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.194:36233
I20260809 23:03:55.633847 15564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.194:36233 every 8 connection(s)
I20260809 23:03:55.635191 15417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/data/info.pb
I20260809 23:03:55.643297 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 15417
I20260809 23:03:55.643496 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/wal/instance
I20260809 23:03:55.643977 24703 external_mini_cluster.cc:1428] Setting key bf3ddfa852aef4dfda93434d1675c9a4
I20260809 23:03:55.647063 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.31.195:0
--local_ip_for_outbound_sockets=127.24.31.195
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34047
--builtin_ntp_servers=127.24.31.212:42549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:03:55.647981 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:55.638004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:59891 (local address 127.24.31.254:34047)
0809 23:03:55.638174 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:55.638178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:55.639574 (+ 1396us) server_negotiation.cc:407] Connection header received
0809 23:03:55.641510 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:55.641515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:55.641604 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:55.641716 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:55.642712 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:55.643467 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:55.645062 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:55.645521 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:55.645652 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:55.647114 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:55.647121 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:55.647126 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:55.647149 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:55.647188 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:55.647209 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:55.647320 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:55.647598 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 23:03:55.647671 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51}
I20260809 23:03:55.649358 15568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:55.638498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:34047 (local address 127.24.31.194:59891)
0809 23:03:55.639427 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:55.639463 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:55.641257 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:55.641934 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:55.641942 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:55.642029 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:55.642554 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:55.642570 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:55.643603 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:55.643611 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:55.644934 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:55.644946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:55.645793 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:55.645800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:55.646062 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:55.646812 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:03:55.646814 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:55.646850 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:03:55.646917 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:55.647378 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:55.647391 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:55.647401 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 23:03:55.647569 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 23:03:55.647969 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":266,"threads_started":1}
I20260809 23:03:55.650753 15566 heartbeater.cc:347] Connected to a master server at 127.24.31.254:34047
I20260809 23:03:55.651039 15566 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:55.651871 15566 heartbeater.cc:511] Master 127.24.31.254:34047 requested a full tablet report, sending...
I20260809 23:03:55.653367 15195 ts_manager.cc:194] Registered new tserver with Master: ca11f079f4114a2389459b33aff8b081 (127.24.31.194:36233)
I20260809 23:03:55.654209 15195 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.194:59891
W20260809 23:03:55.974054 15569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:55.974406 15569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:55.974591 15569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:55.981879 15569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:55.982039 15569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:55.982230 15569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.31.195
I20260809 23:03:55.991340 15569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:42549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.31.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.31.195
--webserver_port=0
--tserver_master_addrs=127.24.31.254:34047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.31.195
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:55.992779 15569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:55.994776 15569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:56.008311 15574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:56.008280 15578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:56.008530 15576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:56.009505 15569 server_base.cc:1034] running on GCE node
I20260809 23:03:56.010185 15569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:56.011111 15569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:56.012379 15569 hybrid_clock.cc:648] HybridClock initialized: now 1786316636012316 us; error 54 us; skew 500 ppm
I20260809 23:03:56.012701 15569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 23:03:56.015120 15569 webserver.cc:460] Webserver started at http://127.24.31.195:39029/ using document root <none> and password file <none>
I20260809 23:03:56.015836 15569 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:56.015976 15569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:56.016397 15569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:56.019452 15569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/data/instance:
uuid: "ff1cd08aed5f47ada5a2084fd6671116"
format_stamp: "Formatted at 2026-08-09 23:03:56 on dist-test-slave-vxj2"
server_key: "4efb3d9092e04edc8a59cdaa9395e714"
server_key_iv: "d941e63647c7e2ac16891e0d81eff71e"
server_key_version: "encryptionkey@0"
I20260809 23:03:56.020231 15569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance:
uuid: "ff1cd08aed5f47ada5a2084fd6671116"
format_stamp: "Formatted at 2026-08-09 23:03:56 on dist-test-slave-vxj2"
server_key: "4efb3d9092e04edc8a59cdaa9395e714"
server_key_iv: "d941e63647c7e2ac16891e0d81eff71e"
server_key_version: "encryptionkey@0"
I20260809 23:03:56.025414 15569 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 23:03:56.028825 15585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:56.030292 15569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:03:56.030443 15569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/wal
uuid: "ff1cd08aed5f47ada5a2084fd6671116"
format_stamp: "Formatted at 2026-08-09 23:03:56 on dist-test-slave-vxj2"
server_key: "4efb3d9092e04edc8a59cdaa9395e714"
server_key_iv: "d941e63647c7e2ac16891e0d81eff71e"
server_key_version: "encryptionkey@0"
I20260809 23:03:56.030619 15569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:56.045801 15569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:56.046605 15569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:56.046917 15569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:56.047873 15569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:03:56.049337 15569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:03:56.049424 15569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:56.049569 15569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:03:56.049659 15569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:56.083179 15569 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.195:46163
I20260809 23:03:56.083279 15711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.195:46163 every 8 connection(s)
I20260809 23:03:56.084501 15569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/data/info.pb
I20260809 23:03:56.086226 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 15569
I20260809 23:03:56.086407 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/wal/instance
I20260809 23:03:56.086882 24703 external_mini_cluster.cc:1428] Setting key 64d117bab8ca64f6a073e780b9bfcd3e
I20260809 23:03:56.097337 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.087269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.195:43627 (local address 127.24.31.254:34047)
0809 23:03:56.087488 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.087495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.089073 (+ 1578us) server_negotiation.cc:407] Connection header received
0809 23:03:56.090908 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.090912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.090981 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.091082 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:56.092084 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.092999 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.094676 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.095140 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.095243 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.096510 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:56.096514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:56.096517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:56.096534 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:56.096566 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:56.096578 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:56.096647 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.096875 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.096950 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 23:03:56.098189 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.087835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:34047 (local address 127.24.31.195:43627)
0809 23:03:56.088922 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.088977 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:56.090639 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:56.091286 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:56.091293 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:56.091372 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:03:56.091922 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.091938 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.093137 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.093140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.094502 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.094514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.095364 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.095369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.095558 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.096215 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:03:56.096217 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:03:56.096265 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:03:56.096334 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:03:56.096671 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:03:56.096679 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:03:56.096684 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:03:56.096845 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 23:03:56.097104 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":380,"threads_started":1}
I20260809 23:03:56.099519 15712 heartbeater.cc:347] Connected to a master server at 127.24.31.254:34047
I20260809 23:03:56.099874 15712 heartbeater.cc:464] Registering TS with master...
I20260809 23:03:56.100898 15712 heartbeater.cc:511] Master 127.24.31.254:34047 requested a full tablet report, sending...
I20260809 23:03:56.102576 15194 ts_manager.cc:194] Registered new tserver with Master: ff1cd08aed5f47ada5a2084fd6671116 (127.24.31.195:46163)
I20260809 23:03:56.103415 15194 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.31.195:43627
I20260809 23:03:56.114990 24703 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:03:56.125551 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.119409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.24.31.254:34047)
0809 23:03:56.119658 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.119669 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.120160 (+ 491us) server_negotiation.cc:407] Connection header received
0809 23:03:56.120354 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.120358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.120430 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.120524 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:56.121080 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.121829 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.123358 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.123810 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.123911 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.124862 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:56.124867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:56.124870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:56.124887 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:56.124916 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:56.124928 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:56.124995 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.125113 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.125234 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 23:03:56.129106 15194 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:03:56.136328 15194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:03:56.165642 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.148632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:46163 (local address 127.0.0.1:59296)
0809 23:03:56.150786 (+ 2154us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.150805 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:56.152040 (+ 1235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:56.153232 (+ 1192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:56.153241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:56.154626 (+ 1385us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:56.155162 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.155180 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.159088 (+ 3908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.159094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.162124 (+ 3030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.162142 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.164565 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.164572 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.164814 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.164874 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 23:03:56.165008 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 23:03:56.165248 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2020,"thread_start_us":1863,"threads_started":1}
I20260809 23:03:56.165993 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.148421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.194:36233 (local address 127.0.0.1:55630)
0809 23:03:56.149092 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.149132 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:56.149345 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:56.159594 (+ 10249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:56.159599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:56.159630 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:56.160318 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.160334 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.162019 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.162024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.163800 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.163812 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.165097 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.165104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.165304 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.165346 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 23:03:56.165434 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 23:03:56.165560 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":162,"thread_start_us":1243,"threads_started":1}
I20260809 23:03:56.166424 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.149756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59296 (local address 127.24.31.195:46163)
0809 23:03:56.151827 (+ 2071us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.151840 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.152180 (+ 340us) server_negotiation.cc:407] Connection header received
0809 23:03:56.152303 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.152314 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.152608 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.152839 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:56.155430 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.158884 (+ 3454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.162379 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.163969 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.164135 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.164429 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.165430 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.165595 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1836,"thread_start_us":1662,"threads_started":1}
I20260809 23:03:56.166956 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.149306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55630 (local address 127.24.31.194:36233)
0809 23:03:56.159001 (+ 9695us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.159013 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.159047 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:03:56.159155 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.159162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.159409 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.159610 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:56.160535 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.161856 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.163997 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.164887 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.165035 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.165289 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.166343 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.166560 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9515,"thread_start_us":9387,"threads_started":1}
I20260809 23:03:56.168840 15730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.150945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.0.0.1:42360)
0809 23:03:56.159639 (+ 8694us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.159659 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:56.159787 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:56.160390 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:56.160444 (+ 54us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:56.160478 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:56.160952 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.160967 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.162897 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.162903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.165102 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.165117 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.167952 (+ 2835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.167959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.168166 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.168191 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 23:03:56.168323 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 23:03:56.168453 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8571,"mutex_wait_us":1,"thread_start_us":8418,"threads_started":1}
I20260809 23:03:56.168999 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.151193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.24.31.193:34483)
0809 23:03:56.159286 (+ 8093us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.159294 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.159767 (+ 473us) server_negotiation.cc:407] Connection header received
0809 23:03:56.159952 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.159961 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.160169 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.160364 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:56.161197 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.162740 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.165380 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.167385 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.167533 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.167809 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.168398 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.168562 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7941,"thread_start_us":7824,"threads_started":1}
I20260809 23:03:56.170480 15641 tablet_service.cc:1467] Processing CreateTablet for tablet f92a6e6c1d1a494495c3cfff3a8cdc61 (DEFAULT_TABLE table=test-table [id=765270a715564ced826b41013226f676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:56.170488 15491 tablet_service.cc:1467] Processing CreateTablet for tablet f92a6e6c1d1a494495c3cfff3a8cdc61 (DEFAULT_TABLE table=test-table [id=765270a715564ced826b41013226f676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:56.171957 15491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f92a6e6c1d1a494495c3cfff3a8cdc61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:56.171957 15641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f92a6e6c1d1a494495c3cfff3a8cdc61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:56.174088 15412 heartbeater.cc:503] Master 127.24.31.254:34047 was elected leader, sending a full tablet report...
I20260809 23:03:56.176432 15340 tablet_service.cc:1467] Processing CreateTablet for tablet f92a6e6c1d1a494495c3cfff3a8cdc61 (DEFAULT_TABLE table=test-table [id=765270a715564ced826b41013226f676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:03:56.177724 15340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f92a6e6c1d1a494495c3cfff3a8cdc61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:56.190897 15732 tablet_bootstrap.cc:492] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: Bootstrap starting.
I20260809 23:03:56.193239 15733 tablet_bootstrap.cc:492] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: Bootstrap starting.
I20260809 23:03:56.194156 15734 tablet_bootstrap.cc:492] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: Bootstrap starting.
I20260809 23:03:56.195998 15732 tablet_bootstrap.cc:654] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:56.197011 15734 tablet_bootstrap.cc:654] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:56.197422 15732 log.cc:824] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:56.197999 15733 tablet_bootstrap.cc:654] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:56.198099 15734 log.cc:824] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:56.199352 15733 log.cc:824] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:56.200902 15732 tablet_bootstrap.cc:492] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: No bootstrap required, opened a new log
I20260809 23:03:56.201241 15732 ts_tablet_manager.cc:1397] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260809 23:03:56.202241 15734 tablet_bootstrap.cc:492] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: No bootstrap required, opened a new log
I20260809 23:03:56.202512 15734 ts_tablet_manager.cc:1397] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 23:03:56.203934 15733 tablet_bootstrap.cc:492] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: No bootstrap required, opened a new log
I20260809 23:03:56.204173 15733 ts_tablet_manager.cc:1397] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260809 23:03:56.209815 15734 raft_consensus.cc:348] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } }
I20260809 23:03:56.210201 15734 raft_consensus.cc:374] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:56.210345 15734 raft_consensus.cc:729] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7b110ac06e74e9bbced530d42d5d0ad, State: Initialized, Role: FOLLOWER
I20260809 23:03:56.211040 15734 consensus_queue.cc:260] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } }
I20260809 23:03:56.211767 15732 raft_consensus.cc:348] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } }
I20260809 23:03:56.212167 15732 raft_consensus.cc:374] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:56.212275 15732 raft_consensus.cc:729] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1cd08aed5f47ada5a2084fd6671116, State: Initialized, Role: FOLLOWER
I20260809 23:03:56.212893 15734 ts_tablet_manager.cc:1428] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 23:03:56.212939 15732 consensus_queue.cc:260] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } }
I20260809 23:03:56.214453 15712 heartbeater.cc:503] Master 127.24.31.254:34047 was elected leader, sending a full tablet report...
I20260809 23:03:56.214978 15732 ts_tablet_manager.cc:1428] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260809 23:03:56.215415 15733 raft_consensus.cc:348] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } }
I20260809 23:03:56.215794 15733 raft_consensus.cc:374] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:56.215953 15733 raft_consensus.cc:729] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca11f079f4114a2389459b33aff8b081, State: Initialized, Role: FOLLOWER
I20260809 23:03:56.216483 15733 consensus_queue.cc:260] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } }
I20260809 23:03:56.217929 15566 heartbeater.cc:503] Master 127.24.31.254:34047 was elected leader, sending a full tablet report...
I20260809 23:03:56.218547 15733 ts_tablet_manager.cc:1428] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
W20260809 23:03:56.336310 15714 tablet.cc:2369] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:03:56.387523 15567 tablet.cc:2369] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:03:56.398878 15413 tablet.cc:2369] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:03:56.418460 15741 raft_consensus.cc:482] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:03:56.418771 15741 raft_consensus.cc:504] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } }
I20260809 23:03:56.420354 15741 leader_election.cc:288] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff1cd08aed5f47ada5a2084fd6671116 (127.24.31.195:46163), c7b110ac06e74e9bbced530d42d5d0ad (127.24.31.193:34483)
I20260809 23:03:56.429369 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.421995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:60111 (local address 127.24.31.193:34483)
0809 23:03:56.422187 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.422193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.422414 (+ 221us) server_negotiation.cc:407] Connection header received
0809 23:03:56.422606 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.422609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.422675 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.422748 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:56.423252 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.424162 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.426815 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.428096 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.428281 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.428350 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.428942 (+ 592us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.429071 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 23:03:56.429394 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.420974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.31.194:58379 (local address 127.24.31.195:46163)
0809 23:03:56.421214 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.421220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.421645 (+ 425us) server_negotiation.cc:407] Connection header received
0809 23:03:56.421912 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.421918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.422012 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.422126 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:03:56.422890 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.423888 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.426823 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.428094 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.428284 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.428377 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.428942 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.429067 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 23:03:56.429644 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.421661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.24.31.194:60111)
0809 23:03:56.422312 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.422333 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:56.422485 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:56.422795 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:56.422801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:56.422830 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:56.423098 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.423105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.424286 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.424291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.426540 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.426555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.428392 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.428400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.428754 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.428820 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 23:03:56.428895 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 23:03:56.429116 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":160,"spinlock_wait_cycles":7424,"thread_start_us":369,"threads_started":1}
I20260809 23:03:56.429893 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.420727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.195:46163 (local address 127.24.31.194:58379)
0809 23:03:56.421535 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.421557 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 23:03:56.421702 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:03:56.422246 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:03:56.422251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:03:56.422275 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:03:56.422707 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.422746 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.424032 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.424036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.426549 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:03:56.426557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.428401 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.428407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:03:56.428749 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.428784 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 23:03:56.428891 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 23:03:56.429116 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":310,"spinlock_wait_cycles":27008,"thread_start_us":393,"threads_started":1}
I20260809 23:03:56.430917 15361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f92a6e6c1d1a494495c3cfff3a8cdc61" candidate_uuid: "ca11f079f4114a2389459b33aff8b081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" is_pre_election: true
I20260809 23:03:56.430917 15662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f92a6e6c1d1a494495c3cfff3a8cdc61" candidate_uuid: "ca11f079f4114a2389459b33aff8b081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1cd08aed5f47ada5a2084fd6671116" is_pre_election: true
I20260809 23:03:56.431536 15361 raft_consensus.cc:2454] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca11f079f4114a2389459b33aff8b081 in term 0.
I20260809 23:03:56.431535 15662 raft_consensus.cc:2454] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca11f079f4114a2389459b33aff8b081 in term 0.
I20260809 23:03:56.432308 15434 leader_election.cc:302] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7b110ac06e74e9bbced530d42d5d0ad, ca11f079f4114a2389459b33aff8b081; no voters:
I20260809 23:03:56.432708 15741 raft_consensus.cc:2781] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:03:56.432910 15741 raft_consensus.cc:482] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:03:56.433022 15741 raft_consensus.cc:3037] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:56.435343 15741 raft_consensus.cc:504] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } }
I20260809 23:03:56.435987 15741 leader_election.cc:288] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [CANDIDATE]: Term 1 election: Requested vote from peers ff1cd08aed5f47ada5a2084fd6671116 (127.24.31.195:46163), c7b110ac06e74e9bbced530d42d5d0ad (127.24.31.193:34483)
I20260809 23:03:56.436476 15662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f92a6e6c1d1a494495c3cfff3a8cdc61" candidate_uuid: "ca11f079f4114a2389459b33aff8b081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1cd08aed5f47ada5a2084fd6671116"
I20260809 23:03:56.436555 15361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f92a6e6c1d1a494495c3cfff3a8cdc61" candidate_uuid: "ca11f079f4114a2389459b33aff8b081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7b110ac06e74e9bbced530d42d5d0ad"
I20260809 23:03:56.436694 15662 raft_consensus.cc:3037] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:56.436791 15361 raft_consensus.cc:3037] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:56.440021 15662 raft_consensus.cc:2454] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca11f079f4114a2389459b33aff8b081 in term 1.
I20260809 23:03:56.440136 15361 raft_consensus.cc:2454] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca11f079f4114a2389459b33aff8b081 in term 1.
I20260809 23:03:56.440444 15434 leader_election.cc:302] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca11f079f4114a2389459b33aff8b081, ff1cd08aed5f47ada5a2084fd6671116; no voters:
I20260809 23:03:56.440701 15741 raft_consensus.cc:2781] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:56.441143 15741 raft_consensus.cc:686] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 1 LEADER]: Becoming Leader. State: Replica: ca11f079f4114a2389459b33aff8b081, State: Running, Role: LEADER
I20260809 23:03:56.441720 15741 consensus_queue.cc:237] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } }
I20260809 23:03:56.446722 15194 catalog_manager.cc:5495] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca11f079f4114a2389459b33aff8b081 (127.24.31.194). New cstate: current_term: 1 leader_uuid: "ca11f079f4114a2389459b33aff8b081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca11f079f4114a2389459b33aff8b081" member_type: VOTER last_known_addr { host: "127.24.31.194" port: 36233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 } health_report { overall_health: UNKNOWN } } }
I20260809 23:03:56.527724 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.522108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.24.31.194:36233)
0809 23:03:56.522443 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.522448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.522469 (+ 21us) server_negotiation.cc:407] Connection header received
0809 23:03:56.522563 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.522567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.522673 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.522917 (+ 244us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:03:56.523340 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.524074 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.525495 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.525950 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.526056 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.526508 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:03:56.527145 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:03:56.527279 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.527454 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.527520 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223}
I20260809 23:03:56.536249 15662 raft_consensus.cc:1261] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [term 1 FOLLOWER]: Refusing update from remote peer ca11f079f4114a2389459b33aff8b081: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:56.536249 15361 raft_consensus.cc:1261] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [term 1 FOLLOWER]: Refusing update from remote peer ca11f079f4114a2389459b33aff8b081: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:03:56.537253 15744 consensus_queue.cc:1035] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1cd08aed5f47ada5a2084fd6671116" member_type: VOTER last_known_addr { host: "127.24.31.195" port: 46163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:56.537654 15741 consensus_queue.cc:1035] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7b110ac06e74e9bbced530d42d5d0ad" member_type: VOTER last_known_addr { host: "127.24.31.193" port: 34483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:03:56.550616 15750 mvcc.cc:205] Tried to move back new op lower bound from 7316752943241670656 to 7316752942869102592. Current Snapshot: MvccSnapshot[applied={T|T < 7316752943241670656}]
I20260809 23:03:56.553086 15751 mvcc.cc:205] Tried to move back new op lower bound from 7316752943241670656 to 7316752942869102592. Current Snapshot: MvccSnapshot[applied={T|T < 7316752943241670656}]
I20260809 23:03:56.564735 15195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:56.565032 15195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:03:56.568992 15195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ccf8843edfd40379b2ab566a88551bc: Sending DeleteTablet for 3 replicas of tablet f92a6e6c1d1a494495c3cfff3a8cdc61
I20260809 23:03:56.570191 15491 tablet_service.cc:1514] Processing DeleteTablet for tablet f92a6e6c1d1a494495c3cfff3a8cdc61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:56 UTC) from {username='slave'} at 127.0.0.1:55630
I20260809 23:03:56.570461 15641 tablet_service.cc:1514] Processing DeleteTablet for tablet f92a6e6c1d1a494495c3cfff3a8cdc61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:56 UTC) from {username='slave'} at 127.0.0.1:59296
I20260809 23:03:56.570490 15340 tablet_service.cc:1514] Processing DeleteTablet for tablet f92a6e6c1d1a494495c3cfff3a8cdc61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:03:56 UTC) from {username='slave'} at 127.0.0.1:42360
I20260809 23:03:56.572548 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 15266
I20260809 23:03:56.582062 15764 tablet_replica.cc:330] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: stopping tablet replica
I20260809 23:03:56.582535 15764 raft_consensus.cc:2229] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:56.582897 15762 tablet_replica.cc:330] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: stopping tablet replica
I20260809 23:03:56.583081 15764 raft_consensus.cc:2258] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:56.583489 15762 raft_consensus.cc:2229] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:03:56.584393 15762 raft_consensus.cc:2258] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:56.585712 15764 ts_tablet_manager.cc:1905] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:56.594101 15762 ts_tablet_manager.cc:1905] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:56.594823 15731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.574117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.24.31.193:34483)
0809 23:03:56.574295 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.574302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.585323 (+ 11021us) server_negotiation.cc:407] Connection header received
0809 23:03:56.585662 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.585671 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.585817 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.585911 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:56.586857 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.588221 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.590251 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.590964 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.591059 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.593119 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:56.593129 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:56.593145 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:56.593869 (+ 724us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:56.593930 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:56.593953 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:56.594027 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.594367 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.594476 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 23:03:56.592906 15763 tablet_replica.cc:330] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: stopping tablet replica
I20260809 23:03:56.595961 15763 raft_consensus.cc:2229] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:03:56.596496 15763 raft_consensus.cc:2258] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:03:56.597476 15406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:56.602038 15762 ts_tablet_manager.cc:1918] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:56.602224 15762 log.cc:1196] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-1/wal/wals/f92a6e6c1d1a494495c3cfff3a8cdc61
I20260809 23:03:56.602826 15762 ts_tablet_manager.cc:1939] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ca11f079f4114a2389459b33aff8b081: Deleting consensus metadata
I20260809 23:03:56.604701 15181 catalog_manager.cc:4841] TS ca11f079f4114a2389459b33aff8b081 (127.24.31.194:36233): tablet f92a6e6c1d1a494495c3cfff3a8cdc61 (table test-table [id=765270a715564ced826b41013226f676]) successfully deleted
I20260809 23:03:56.605930 15763 ts_tablet_manager.cc:1905] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:03:56.612865 15763 ts_tablet_manager.cc:1918] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:56.613036 15763 log.cc:1196] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-2/wal/wals/f92a6e6c1d1a494495c3cfff3a8cdc61
I20260809 23:03:56.615477 15763 ts_tablet_manager.cc:1939] T f92a6e6c1d1a494495c3cfff3a8cdc61 P ff1cd08aed5f47ada5a2084fd6671116: Deleting consensus metadata
I20260809 23:03:56.617834 15178 catalog_manager.cc:4841] TS ff1cd08aed5f47ada5a2084fd6671116 (127.24.31.195:46163): tablet f92a6e6c1d1a494495c3cfff3a8cdc61 (table test-table [id=765270a715564ced826b41013226f676]) successfully deleted
I20260809 23:03:56.723582 15764 ts_tablet_manager.cc:1918] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:03:56.723857 15764 log.cc:1196] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: Deleting WAL directory at /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316311377261-24703-0/minicluster-data/ts-0/wal/wals/f92a6e6c1d1a494495c3cfff3a8cdc61
I20260809 23:03:56.724526 15764 ts_tablet_manager.cc:1939] T f92a6e6c1d1a494495c3cfff3a8cdc61 P c7b110ac06e74e9bbced530d42d5d0ad: Deleting consensus metadata
I20260809 23:03:56.724521 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 15266
I20260809 23:03:56.745079 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 15417
W20260809 23:03:56.745078 15178 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.31.193:34483 (error 108)
I20260809 23:03:56.749938 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.746650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.0.0.1:42370)
0809 23:03:56.749576 (+ 2926us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.749721 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2798,"thread_start_us":2682,"threads_started":1}
W20260809 23:03:56.750243 15178 catalog_manager.cc:4584] TS c7b110ac06e74e9bbced530d42d5d0ad (127.24.31.193:34483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f92a6e6c1d1a494495c3cfff3a8cdc61: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34483: connect: Connection refused (error 111)
I20260809 23:03:56.754093 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.746340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55654 (local address 127.24.31.194:36233)
0809 23:03:56.746586 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.746592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.746612 (+ 20us) server_negotiation.cc:407] Connection header received
0809 23:03:56.746683 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.746692 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.746798 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.746929 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:56.747792 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.749365 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.750837 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.751349 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.751500 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.752567 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:56.752576 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:56.752589 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:56.753322 (+ 733us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:56.753368 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:56.753393 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:56.753510 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.753662 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.753782 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 23:03:56.754763 15560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:56.803824 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.803246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.0.0.1:42384)
0809 23:03:56.803522 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.803646 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 23:03:56.804939 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.804503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.0.0.1:42392)
0809 23:03:56.804667 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.804768 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 23:03:56.865728 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.865267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.0.0.1:42408)
0809 23:03:56.865476 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.865570 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 23:03:56.866904 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.866556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.0.0.1:42424)
0809 23:03:56.866683 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.866751 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 23:03:56.871824 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 15417
I20260809 23:03:56.893460 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 15569
I20260809 23:03:56.900436 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.894233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59302 (local address 127.24.31.195:46163)
0809 23:03:56.894422 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:56.894430 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:56.894514 (+ 84us) server_negotiation.cc:407] Connection header received
0809 23:03:56.894726 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:56.894732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:56.894871 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:56.894987 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:56.895403 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.896129 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.897455 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:56.897864 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:56.898010 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:56.899072 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:56.899081 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:56.899095 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:03:56.899652 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:56.899710 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:03:56.899735 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:56.899867 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:56.900006 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 23:03:56.900114 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 23:03:56.901118 15706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:56.947366 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.946734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.0.0.1:42434)
0809 23:03:56.947064 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.947176 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260809 23:03:56.948588 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:56.948133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.0.0.1:42438)
0809 23:03:56.948354 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:56.948436 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260809 23:03:56.991326 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 15569
I20260809 23:03:57.011250 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 15158
I20260809 23:03:57.012424 15230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:57.123672 15768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:57.123224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.193:34483 (local address 127.0.0.1:42442)
0809 23:03:57.123393 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 23:03:57.123503 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.193:34483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 23:03:57.123718 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 15158
2026-08-09T23:03:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3071 ms)
[----------] 6 tests from EncryptionPolicyTest (22390 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:03:57 dist-test-slave-vxj2 krb5kdc[15774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:03:57 dist-test-slave-vxj2 krb5kdc[15774](info): set up 2 sockets
Aug 09 23:03:57 dist-test-slave-vxj2 krb5kdc[15774](info): commencing operation
krb5kdc: starting...
W20260809 23:03:58.321264 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:03:58 dist-test-slave-vxj2 krb5kdc[15774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:03:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:03:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:03:58.601438 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:03:58.933967 15788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:03:58.934350 15788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:03:58.934517 15788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:03:58.941609 15788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:03:58.941748 15788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:03:58.941852 15788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:03:58.941975 15788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:03:58.942059 15788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:03:58.951515 15788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35441
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:03:58.952795 15788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:03:58.954098 15788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:03:58.966029 15793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:58.966181 15794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:03:58.966151 15796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:03:58.967094 15788 server_base.cc:1034] running on GCE node
I20260809 23:03:58.967824 15788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:03:58.969251 15788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:03:58.970471 15788 hybrid_clock.cc:648] HybridClock initialized: now 1786316638970416 us; error 52 us; skew 500 ppm
I20260809 23:03:58.970846 15788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:03:58 dist-test-slave-vxj2 krb5kdc[15774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316638, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:03:58.974195 15788 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:03:58.975589 15788 webserver.cc:460] Webserver started at http://127.0.0.1:37593/ using document root <none> and password file <none>
I20260809 23:03:58.976208 15788 fs_manager.cc:359] Metadata directory not provided
I20260809 23:03:58.976307 15788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:03:58.976667 15788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:03:58.980042 15788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "17452d2ba1ca4f288cbd7904f35b0634"
format_stamp: "Formatted at 2026-08-09 23:03:58 on dist-test-slave-vxj2"
server_key: "8224de13231a0fb55960412aaa2302aa"
server_key_iv: "b11527428da470da49495c2743cd0df4"
server_key_version: "encryptionkey@0"
I20260809 23:03:58.981030 15788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "17452d2ba1ca4f288cbd7904f35b0634"
format_stamp: "Formatted at 2026-08-09 23:03:58 on dist-test-slave-vxj2"
server_key: "8224de13231a0fb55960412aaa2302aa"
server_key_iv: "b11527428da470da49495c2743cd0df4"
server_key_version: "encryptionkey@0"
I20260809 23:03:58.986524 15788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:03:58.990026 15807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:03:58.991297 15788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:03:58.991477 15788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "17452d2ba1ca4f288cbd7904f35b0634"
format_stamp: "Formatted at 2026-08-09 23:03:58 on dist-test-slave-vxj2"
server_key: "8224de13231a0fb55960412aaa2302aa"
server_key_iv: "b11527428da470da49495c2743cd0df4"
server_key_version: "encryptionkey@0"
I20260809 23:03:58.991667 15788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:03:59.013968 15788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:03:59.018512 15788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:03:59.018870 15788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:03:59.037637 15788 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35441
I20260809 23:03:59.037671 15876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35441 every 8 connection(s)
I20260809 23:03:59.038952 15788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:03:59.039472 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 15788
I20260809 23:03:59.039742 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:03:59.040598 24703 external_mini_cluster.cc:1428] Setting key a80ef4390930259f734a6b0080092880
I20260809 23:03:59.043349 15877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:03:59.054122 15877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634: Bootstrap starting.
Aug 09 23:03:59 dist-test-slave-vxj2 krb5kdc[15774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:03:59.057755 15877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634: Neither blocks nor log segments found. Creating new log.
I20260809 23:03:59.058835 15877 log.cc:824] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634: Log is configured to *not* fsync() on all Append() calls
I20260809 23:03:59.061863 15877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634: No bootstrap required, opened a new log
I20260809 23:03:59.063867 15881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:59.042632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57677 (local address 127.0.0.1:35441)
0809 23:03:59.043375 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:59.043386 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:59.043458 (+ 72us) server_negotiation.cc:407] Connection header received
0809 23:03:59.045111 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:59.045131 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:59.045552 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:59.045936 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:59.047404 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:59.048690 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:59.050048 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:59.050719 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:59.050925 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:59.055696 (+ 4771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:59.055721 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:59.055735 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:59.055786 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:59.059216 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:59.060082 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:59.060087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:59.060093 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:59.060254 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:59.060836 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:59.060841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:59.060843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:59.061525 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:59.061845 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:59.062365 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 23:03:59.062787 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":331,"threads_started":1}
I20260809 23:03:59.068071 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:03:59.070433 15877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17452d2ba1ca4f288cbd7904f35b0634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } }
I20260809 23:03:59.070806 15877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:03:59.070931 15877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17452d2ba1ca4f288cbd7904f35b0634, State: Initialized, Role: FOLLOWER
I20260809 23:03:59.071519 15877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17452d2ba1ca4f288cbd7904f35b0634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } }
I20260809 23:03:59.071753 15877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:03:59.071900 15877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:03:59.072074 15877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:03:59.075277 15877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17452d2ba1ca4f288cbd7904f35b0634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } }
I20260809 23:03:59.075862 15877 leader_election.cc:302] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17452d2ba1ca4f288cbd7904f35b0634; no voters:
I20260809 23:03:59.076553 15877 leader_election.cc:288] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:03:59.076683 15890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:03:59.077695 15890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [term 1 LEADER]: Becoming Leader. State: Replica: 17452d2ba1ca4f288cbd7904f35b0634, State: Running, Role: LEADER
I20260809 23:03:59.078421 15890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17452d2ba1ca4f288cbd7904f35b0634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } }
I20260809 23:03:59.078934 15877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:03:59.082091 15891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17452d2ba1ca4f288cbd7904f35b0634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17452d2ba1ca4f288cbd7904f35b0634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } } }
I20260809 23:03:59.082556 15891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:59.082157 15892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17452d2ba1ca4f288cbd7904f35b0634. Latest consensus state: current_term: 1 leader_uuid: "17452d2ba1ca4f288cbd7904f35b0634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17452d2ba1ca4f288cbd7904f35b0634" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } } }
I20260809 23:03:59.082757 15892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634 [sys.catalog]: This master's current role is: LEADER
I20260809 23:03:59.083441 15895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:03:59.089318 15895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:03:59.093950 15881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:03:59.072994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40251 (local address 127.0.0.1:35441)
0809 23:03:59.073229 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 23:03:59.073237 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:03:59.073742 (+ 505us) server_negotiation.cc:407] Connection header received
0809 23:03:59.073900 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:03:59.073904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:03:59.073960 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:03:59.074038 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:03:59.075963 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:59.077304 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:59.079059 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:03:59.079841 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:03:59.080027 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:03:59.083785 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:03:59.083792 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:03:59.083800 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:03:59.083849 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:03:59.088728 (+ 4879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:59.089977 (+ 1249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:59.089986 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:59.089992 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:59.090254 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:03:59.090968 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:03:59.090975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:03:59.090981 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:03:59.091462 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:03:59.091812 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0809 23:03:59.093113 (+ 1301us) server_negotiation.cc:299] Negotiation successful
0809 23:03:59.093387 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":60}
I20260809 23:03:59.105610 15895 catalog_manager.cc:1347] Generated new cluster ID: df17cea5a3a24d5ca2a931c44dba5591
I20260809 23:03:59.105754 15895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:03:59.137243 15895 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:03:59.138556 15895 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:03:59.155072 15895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17452d2ba1ca4f288cbd7904f35b0634: Generated new TSK 0
I20260809 23:03:59.155911 15895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:03:59.168772 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 15788
I20260809 23:03:59.170243 15871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:03:59.266199 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 15788
W20260809 23:03:59.284333 15772 connection.cc:535] client connection to 127.0.0.1:35441 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35441): Connection reset by peer (error 104)
2026-08-09T23:03:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2130 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:03:59 dist-test-slave-vxj2 krb5kdc[15916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:03:59 dist-test-slave-vxj2 krb5kdc[15916](info): set up 2 sockets
Aug 09 23:03:59 dist-test-slave-vxj2 krb5kdc[15916](info): commencing operation
krb5kdc: starting...
W20260809 23:04:00.451664 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:00 dist-test-slave-vxj2 krb5kdc[15916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:00.744452 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:01.076886 15930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:01.077284 15930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:01.077456 15930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:01.084957 15930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:01.085142 15930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:01.085305 15930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:01.085443 15930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:01.085532 15930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:01.095829 15930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:01.097229 15930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:01.098594 15930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:01.110684 15935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:01.111001 15936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:01.111083 15938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:01.112088 15930 server_base.cc:1034] running on GCE node
I20260809 23:04:01.112932 15930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:01.114362 15930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:01.115568 15930 hybrid_clock.cc:648] HybridClock initialized: now 1786316641115528 us; error 27 us; skew 500 ppm
I20260809 23:04:01.115959 15930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:01 dist-test-slave-vxj2 krb5kdc[15916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:01.124094 15930 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:01.125751 15930 webserver.cc:460] Webserver started at http://127.0.0.1:45313/ using document root <none> and password file <none>
I20260809 23:04:01.126477 15930 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:01.126632 15930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:01.127031 15930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:01.130369 15930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "da28ad103b864ab2b80eb24b46af683b"
format_stamp: "Formatted at 2026-08-09 23:04:01 on dist-test-slave-vxj2"
server_key: "6a84e7c36a8585944ccdbe77dcc91f07"
server_key_iv: "989b48180b9b4e5c024b94eb9968b013"
server_key_version: "encryptionkey@0"
I20260809 23:04:01.131168 15930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "da28ad103b864ab2b80eb24b46af683b"
format_stamp: "Formatted at 2026-08-09 23:04:01 on dist-test-slave-vxj2"
server_key: "6a84e7c36a8585944ccdbe77dcc91f07"
server_key_iv: "989b48180b9b4e5c024b94eb9968b013"
server_key_version: "encryptionkey@0"
I20260809 23:04:01.136909 15930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 23:04:01.140583 15949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:01.142045 15930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:04:01.142247 15930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "da28ad103b864ab2b80eb24b46af683b"
format_stamp: "Formatted at 2026-08-09 23:04:01 on dist-test-slave-vxj2"
server_key: "6a84e7c36a8585944ccdbe77dcc91f07"
server_key_iv: "989b48180b9b4e5c024b94eb9968b013"
server_key_version: "encryptionkey@0"
I20260809 23:04:01.142463 15930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:01.166829 15930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:01.171305 15930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:01.171602 15930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:01.190759 15930 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42459
I20260809 23:04:01.190794 16010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42459 every 8 connection(s)
I20260809 23:04:01.192106 15930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:01.195771 16011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:01.200254 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 15930
I20260809 23:04:01.200460 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:01.200928 24703 external_mini_cluster.cc:1428] Setting key 40aecde940afafbe66e7945df6e3352d
I20260809 23:04:01.206660 16011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b: Bootstrap starting.
I20260809 23:04:01.210218 16011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:01.211428 16011 log.cc:824] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:01.214464 16011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b: No bootstrap required, opened a new log
I20260809 23:04:01.214856 16014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:01.202689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33117 (local address 127.0.0.1:42459)
0809 23:04:01.203487 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:01.203501 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:01.203542 (+ 41us) server_negotiation.cc:407] Connection header received
0809 23:04:01.205295 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:01.205336 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:01.205697 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:01.206015 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:01.206698 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:01.208123 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:01.210017 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:01.210844 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:01.210985 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:01.212386 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:01.212399 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:01.212418 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:01.213104 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:01.213220 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:01.213247 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:01.213373 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:01.213657 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 23:04:01.213885 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":354,"threads_started":1}
I20260809 23:04:01.218587 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:01.224270 16011 raft_consensus.cc:348] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da28ad103b864ab2b80eb24b46af683b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260809 23:04:01.224661 16011 raft_consensus.cc:374] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:01.224794 16011 raft_consensus.cc:729] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da28ad103b864ab2b80eb24b46af683b, State: Initialized, Role: FOLLOWER
I20260809 23:04:01.225519 16011 consensus_queue.cc:260] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da28ad103b864ab2b80eb24b46af683b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260809 23:04:01.225852 16011 raft_consensus.cc:388] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:01.226016 16011 raft_consensus.cc:482] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:01.226205 16011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:01.229425 16011 raft_consensus.cc:504] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da28ad103b864ab2b80eb24b46af683b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260809 23:04:01.230041 16011 leader_election.cc:302] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da28ad103b864ab2b80eb24b46af683b; no voters:
I20260809 23:04:01.230773 16011 leader_election.cc:288] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:01.230930 16022 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:01.231557 16014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:01.223898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58501 (local address 127.0.0.1:42459)
0809 23:04:01.224075 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:01.224080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:01.224616 (+ 536us) server_negotiation.cc:407] Connection header received
0809 23:04:01.224864 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:01.224868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:01.224926 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:01.225043 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:01.225697 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:01.226655 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:01.228536 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:01.228974 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:01.229078 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:01.230447 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:01.230456 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:01.230461 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:01.230487 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:01.230540 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:01.230563 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:01.230856 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:01.230943 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 23:04:01.230998 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 23:04:01.232510 16022 raft_consensus.cc:686] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [term 1 LEADER]: Becoming Leader. State: Replica: da28ad103b864ab2b80eb24b46af683b, State: Running, Role: LEADER
I20260809 23:04:01.233201 16022 consensus_queue.cc:237] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da28ad103b864ab2b80eb24b46af683b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260809 23:04:01.233572 16011 sys_catalog.cc:564] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:01.236186 16023 sys_catalog.cc:455] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da28ad103b864ab2b80eb24b46af683b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da28ad103b864ab2b80eb24b46af683b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } } }
I20260809 23:04:01.236579 16023 sys_catalog.cc:458] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:01.236292 16024 sys_catalog.cc:455] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [sys.catalog]: SysCatalogTable state changed. Reason: New leader da28ad103b864ab2b80eb24b46af683b. Latest consensus state: current_term: 1 leader_uuid: "da28ad103b864ab2b80eb24b46af683b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da28ad103b864ab2b80eb24b46af683b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } } }
I20260809 23:04:01.236714 16024 sys_catalog.cc:458] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:01.238683 16027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:01.243928 16027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:01.252862 16027 catalog_manager.cc:1347] Generated new cluster ID: 33aeb7549d084c55b87b8f4194dee5ed
I20260809 23:04:01.253036 16027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:01.265245 16027 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:01.266609 16027 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:01.276283 16027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da28ad103b864ab2b80eb24b46af683b: Generated new TSK 0
I20260809 23:04:01.277094 16027 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:01.283407 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 15930
W20260809 23:04:01.284557 16005 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33117
W20260809 23:04:01.285398 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 15930: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:01.285578 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 15930
W20260809 23:04:01.305286 15914 connection.cc:535] client connection to 127.0.0.1:42459 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42459): Connection reset by peer (error 104)
2026-08-09T23:04:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2021 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:01 dist-test-slave-vxj2 krb5kdc[16045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:01 dist-test-slave-vxj2 krb5kdc[16045](info): set up 2 sockets
Aug 09 23:04:01 dist-test-slave-vxj2 krb5kdc[16045](info): commencing operation
krb5kdc: starting...
W20260809 23:04:02.482175 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:02 dist-test-slave-vxj2 krb5kdc[16045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:02.765386 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:03.095808 16062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:03.096117 16062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:03.096207 16062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:03.103435 16062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:03.103555 16062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:03.103603 16062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:03.103657 16062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:03.103706 16062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:03.113778 16062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:03.115067 16062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:03.116336 16062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:03.128957 16068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:03.128937 16069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:03.128938 16071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:03.129594 16062 server_base.cc:1034] running on GCE node
I20260809 23:04:03.130535 16062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:03.131959 16062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:03.133249 16062 hybrid_clock.cc:648] HybridClock initialized: now 1786316643133174 us; error 59 us; skew 500 ppm
I20260809 23:04:03.133596 16062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:03 dist-test-slave-vxj2 krb5kdc[16045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:03.136906 16062 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:03.138587 16062 webserver.cc:460] Webserver started at http://127.0.0.1:37659/ using document root <none> and password file <none>
I20260809 23:04:03.139281 16062 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:03.139423 16062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:03.139734 16062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:03.142879 16062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "bdadd22ad4034f58ac875df2654dbe90"
format_stamp: "Formatted at 2026-08-09 23:04:03 on dist-test-slave-vxj2"
server_key: "6d65b66fc63d6246f621076089be4766"
server_key_iv: "b238e38dde830f55aba21b4de8d9db6e"
server_key_version: "encryptionkey@0"
I20260809 23:04:03.143633 16062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "bdadd22ad4034f58ac875df2654dbe90"
format_stamp: "Formatted at 2026-08-09 23:04:03 on dist-test-slave-vxj2"
server_key: "6d65b66fc63d6246f621076089be4766"
server_key_iv: "b238e38dde830f55aba21b4de8d9db6e"
server_key_version: "encryptionkey@0"
I20260809 23:04:03.148792 16062 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 23:04:03.152379 16078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:03.153707 16062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:04:03.153932 16062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "bdadd22ad4034f58ac875df2654dbe90"
format_stamp: "Formatted at 2026-08-09 23:04:03 on dist-test-slave-vxj2"
server_key: "6d65b66fc63d6246f621076089be4766"
server_key_iv: "b238e38dde830f55aba21b4de8d9db6e"
server_key_version: "encryptionkey@0"
I20260809 23:04:03.154138 16062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:03.159917 16062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:03.164063 16062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:03.164369 16062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:03.184137 16062 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37119
I20260809 23:04:03.184191 16140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37119 every 8 connection(s)
I20260809 23:04:03.185607 16062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:03.189296 16141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:03.191962 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 16062
I20260809 23:04:03.192183 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:03.192633 24703 external_mini_cluster.cc:1428] Setting key 474f9c45ec17486cdc0b2d4aa3946d4c
I20260809 23:04:03.200332 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:03.194441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47511 (local address 127.0.0.1:37119)
0809 23:04:03.195259 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:03.195270 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:03.195312 (+ 42us) server_negotiation.cc:407] Connection header received
0809 23:04:03.196938 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:03.196960 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:03.197368 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:03.197685 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:03.198843 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:03.198849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:03.198861 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:03.198877 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:03.198939 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:03.198957 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:03.199119 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:03.199247 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 23:04:03.199453 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":369,"threads_started":1}
I20260809 23:04:03.200980 16141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90: Bootstrap starting.
I20260809 23:04:03.204104 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:03.204679 16141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:03.205868 16141 log.cc:824] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:03.209077 16141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90: No bootstrap required, opened a new log
I20260809 23:04:03.211690 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:03.209285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41775 (local address 127.0.0.1:37119)
0809 23:04:03.209475 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:03.209480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:03.209831 (+ 351us) server_negotiation.cc:407] Connection header received
0809 23:04:03.210027 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:03.210030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:03.210086 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:03.210159 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:03.211074 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:03.211078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:03.211081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:03.211097 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:03.211132 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:03.211146 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:03.211244 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:03.211353 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 23:04:03.211400 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 23:04:03.217453 16141 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdadd22ad4034f58ac875df2654dbe90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } }
I20260809 23:04:03.217805 16141 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:03.217914 16141 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdadd22ad4034f58ac875df2654dbe90, State: Initialized, Role: FOLLOWER
I20260809 23:04:03.218448 16141 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdadd22ad4034f58ac875df2654dbe90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } }
I20260809 23:04:03.218699 16141 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:03.218845 16141 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:03.219038 16141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:03.221287 16141 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdadd22ad4034f58ac875df2654dbe90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } }
I20260809 23:04:03.221769 16141 leader_election.cc:302] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdadd22ad4034f58ac875df2654dbe90; no voters:
I20260809 23:04:03.222313 16141 leader_election.cc:288] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:03.222463 16154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:03.223250 16154 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [term 1 LEADER]: Becoming Leader. State: Replica: bdadd22ad4034f58ac875df2654dbe90, State: Running, Role: LEADER
I20260809 23:04:03.223956 16154 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdadd22ad4034f58ac875df2654dbe90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } }
I20260809 23:04:03.224419 16141 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:03.226796 16155 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdadd22ad4034f58ac875df2654dbe90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdadd22ad4034f58ac875df2654dbe90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } } }
I20260809 23:04:03.227216 16155 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:03.227895 16156 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdadd22ad4034f58ac875df2654dbe90. Latest consensus state: current_term: 1 leader_uuid: "bdadd22ad4034f58ac875df2654dbe90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdadd22ad4034f58ac875df2654dbe90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37119 } } }
I20260809 23:04:03.228314 16156 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:03.230324 16160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:03.236490 16160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:03.245606 16160 catalog_manager.cc:1347] Generated new cluster ID: c281a420f2bc4c63a38e0b28b5241928
I20260809 23:04:03.245731 16160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:03.282893 16160 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:03.283345 16160 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:03.289242 16160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdadd22ad4034f58ac875df2654dbe90: Generated new TSK 0
I20260809 23:04:03.290177 16160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:03.308787 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16062
W20260809 23:04:03.309949 16135 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47511
W20260809 23:04:03.310737 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16062: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:03.310904 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 16062
2026-08-09T23:04:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2022 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:03 dist-test-slave-vxj2 krb5kdc[16177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:03 dist-test-slave-vxj2 krb5kdc[16177](info): set up 2 sockets
Aug 09 23:04:03 dist-test-slave-vxj2 krb5kdc[16177](info): commencing operation
krb5kdc: starting...
W20260809 23:04:04.518391 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:04 dist-test-slave-vxj2 krb5kdc[16177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:04.802289 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41967
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:05.138167 16193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:05.138463 16193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:05.138574 16193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:05.145653 16193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:05.145771 16193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:05.145823 16193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:05.145887 16193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:05.145932 16193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:05.155730 16193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:05.156947 16193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:05.158238 16193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:05.169869 16205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:05.169780 16206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:05.169831 16208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:05.170941 16193 server_base.cc:1034] running on GCE node
I20260809 23:04:05.171708 16193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:05.173022 16193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:05.174245 16193 hybrid_clock.cc:648] HybridClock initialized: now 1786316645174188 us; error 70 us; skew 500 ppm
I20260809 23:04:05.174587 16193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:05 dist-test-slave-vxj2 krb5kdc[16177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:05.177773 16193 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:05.179208 16193 webserver.cc:460] Webserver started at http://127.0.0.1:42797/ using document root <none> and password file <none>
I20260809 23:04:05.179834 16193 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:05.179975 16193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:05.180253 16193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:05.183282 16193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "46efe73e577446258d051bff8b263ed1"
format_stamp: "Formatted at 2026-08-09 23:04:05 on dist-test-slave-vxj2"
server_key: "4c8e589ce12f259d8ba701fd5216b5c4"
server_key_iv: "cd33fbd5e6d0929ef6e7cd5d32bf5cc3"
server_key_version: "encryptionkey@0"
I20260809 23:04:05.184084 16193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "46efe73e577446258d051bff8b263ed1"
format_stamp: "Formatted at 2026-08-09 23:04:05 on dist-test-slave-vxj2"
server_key: "4c8e589ce12f259d8ba701fd5216b5c4"
server_key_iv: "cd33fbd5e6d0929ef6e7cd5d32bf5cc3"
server_key_version: "encryptionkey@0"
I20260809 23:04:05.189093 16193 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 23:04:05.193038 16215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:05.194437 16193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:04:05.194622 16193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "46efe73e577446258d051bff8b263ed1"
format_stamp: "Formatted at 2026-08-09 23:04:05 on dist-test-slave-vxj2"
server_key: "4c8e589ce12f259d8ba701fd5216b5c4"
server_key_iv: "cd33fbd5e6d0929ef6e7cd5d32bf5cc3"
server_key_version: "encryptionkey@0"
I20260809 23:04:05.194830 16193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:05.212853 16193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:05.216951 16193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:05.217324 16193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:05.236801 16193 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41967
I20260809 23:04:05.236857 16281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41967 every 8 connection(s)
I20260809 23:04:05.238261 16193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:05.239077 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 16193
I20260809 23:04:05.239307 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:05.239815 24703 external_mini_cluster.cc:1428] Setting key 66a472b6cb050fb7a18d2bd7783c9fee
I20260809 23:04:05.242722 16282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 23:04:05 dist-test-slave-vxj2 krb5kdc[16177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:04:05.256443 16282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1: Bootstrap starting.
I20260809 23:04:05.259745 16282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:05.261063 16282 log.cc:824] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:05.264467 16282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1: No bootstrap required, opened a new log
I20260809 23:04:05.264568 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:05.241615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48059 (local address 127.0.0.1:41967)
0809 23:04:05.242428 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:05.242440 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:05.242483 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:04:05.244188 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:05.244210 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:05.244577 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:05.244907 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:05.246829 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:05.248482 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:05.250402 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:05.251115 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:05.251279 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:05.256632 (+ 5353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:05.256648 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:05.256664 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:05.256711 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:05.259960 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:05.260869 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:05.260875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:05.260881 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:05.261064 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:05.261492 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:05.261496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:05.261498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:05.262339 (+ 841us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:05.262647 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:05.263145 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 23:04:05.263514 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":388,"threads_started":1}
I20260809 23:04:05.268411 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:05.273061 16282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46efe73e577446258d051bff8b263ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41967 } }
I20260809 23:04:05.273553 16282 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:05.273679 16282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46efe73e577446258d051bff8b263ed1, State: Initialized, Role: FOLLOWER
I20260809 23:04:05.274329 16282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46efe73e577446258d051bff8b263ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41967 } }
I20260809 23:04:05.274585 16282 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:05.274734 16282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:05.274931 16282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:05.277328 16282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46efe73e577446258d051bff8b263ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41967 } }
I20260809 23:04:05.277840 16282 leader_election.cc:302] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46efe73e577446258d051bff8b263ed1; no voters:
I20260809 23:04:05.278545 16282 leader_election.cc:288] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:05.278656 16293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:05.279377 16293 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [term 1 LEADER]: Becoming Leader. State: Replica: 46efe73e577446258d051bff8b263ed1, State: Running, Role: LEADER
I20260809 23:04:05.280040 16293 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46efe73e577446258d051bff8b263ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41967 } }
I20260809 23:04:05.281062 16282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:05.288115 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:05.272812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41095 (local address 127.0.0.1:41967)
0809 23:04:05.273012 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:05.273019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:05.273571 (+ 552us) server_negotiation.cc:407] Connection header received
0809 23:04:05.273787 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:05.273793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:05.273875 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:05.273984 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:05.275722 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:05.276511 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:05.277879 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:05.278435 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:05.278583 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:05.281651 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:05.281660 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:05.281669 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:05.281728 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:05.285095 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:05.286002 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:05.286007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:05.286011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:05.286184 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:05.286592 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:05.286600 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:05.286606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:05.287024 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:05.287301 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:05.287553 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 23:04:05.287785 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 23:04:05.289670 16294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46efe73e577446258d051bff8b263ed1. Latest consensus state: current_term: 1 leader_uuid: "46efe73e577446258d051bff8b263ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46efe73e577446258d051bff8b263ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41967 } } }
I20260809 23:04:05.289899 16295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46efe73e577446258d051bff8b263ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46efe73e577446258d051bff8b263ed1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41967 } } }
I20260809 23:04:05.290162 16294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:05.290181 16295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:05.291272 16306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:05.296592 16306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:05.305559 16306 catalog_manager.cc:1347] Generated new cluster ID: f585123ac5264827be18d255b38f50d4
I20260809 23:04:05.305706 16306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:05.315162 16306 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:05.316429 16306 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:05.327318 16306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46efe73e577446258d051bff8b263ed1: Generated new TSK 0
I20260809 23:04:05.328141 16306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:05.337246 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16193
I20260809 23:04:05.338289 16273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:05.432135 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 16193
2026-08-09T23:04:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2120 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:05 dist-test-slave-vxj2 krb5kdc[16319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:05 dist-test-slave-vxj2 krb5kdc[16319](info): set up 2 sockets
Aug 09 23:04:05 dist-test-slave-vxj2 krb5kdc[16319](info): commencing operation
krb5kdc: starting...
W20260809 23:04:06.616564 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:06 dist-test-slave-vxj2 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:06.909530 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40085
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:07.252592 16336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:07.252939 16336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:07.253103 16336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:07.260819 16336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:07.260974 16336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:07.261053 16336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:07.261139 16336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:07.261238 16336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:07.271027 16336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:07.272266 16336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:07.273613 16336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:07.285707 16342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:07.285799 16343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:07.285748 16345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:07.287379 16336 server_base.cc:1034] running on GCE node
I20260809 23:04:07.288136 16336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:07.289551 16336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:07.290788 16336 hybrid_clock.cc:648] HybridClock initialized: now 1786316647290752 us; error 39 us; skew 500 ppm
I20260809 23:04:07.291157 16336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:07 dist-test-slave-vxj2 krb5kdc[16319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316647, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:07.298102 16336 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:07.299700 16336 webserver.cc:460] Webserver started at http://127.0.0.1:33881/ using document root <none> and password file <none>
I20260809 23:04:07.300387 16336 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:07.300534 16336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:07.300830 16336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:07.303867 16336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "8dd6179379544aceb8e8f93d0383a2bd"
format_stamp: "Formatted at 2026-08-09 23:04:07 on dist-test-slave-vxj2"
server_key: "633d9270036022d72d2cd6cdf32da30d"
server_key_iv: "79d08938a7398db53dbda342ed29470e"
server_key_version: "encryptionkey@0"
I20260809 23:04:07.304653 16336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "8dd6179379544aceb8e8f93d0383a2bd"
format_stamp: "Formatted at 2026-08-09 23:04:07 on dist-test-slave-vxj2"
server_key: "633d9270036022d72d2cd6cdf32da30d"
server_key_iv: "79d08938a7398db53dbda342ed29470e"
server_key_version: "encryptionkey@0"
I20260809 23:04:07.309746 16336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 23:04:07.313146 16352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:07.314462 16336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:04:07.314643 16336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "8dd6179379544aceb8e8f93d0383a2bd"
format_stamp: "Formatted at 2026-08-09 23:04:07 on dist-test-slave-vxj2"
server_key: "633d9270036022d72d2cd6cdf32da30d"
server_key_iv: "79d08938a7398db53dbda342ed29470e"
server_key_version: "encryptionkey@0"
I20260809 23:04:07.314832 16336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:07.360291 16336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:07.364449 16336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:07.364740 16336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:07.387876 16336 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40085
I20260809 23:04:07.387916 16415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40085 every 8 connection(s)
I20260809 23:04:07.389338 16336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:07.393062 16416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:07.397488 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 16336
I20260809 23:04:07.397666 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:07.398236 24703 external_mini_cluster.cc:1428] Setting key 4917b85a294a08fd0706fce7d9078927
I20260809 23:04:07.404552 16416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd: Bootstrap starting.
I20260809 23:04:07.408581 16416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:07.409817 16416 log.cc:824] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:07.412974 16416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd: No bootstrap required, opened a new log
I20260809 23:04:07.413386 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:07.400076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50239 (local address 127.0.0.1:40085)
0809 23:04:07.400849 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:07.400860 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:07.400901 (+ 41us) server_negotiation.cc:407] Connection header received
0809 23:04:07.403266 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:07.403291 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:07.403675 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:07.404005 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:07.404628 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:07.406709 (+ 2081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:07.408668 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:07.409438 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:07.409593 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:07.410872 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:07.410884 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:07.410902 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:07.411614 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:07.411696 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:07.411725 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:07.411813 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:07.412073 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 23:04:07.412345 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":384,"threads_started":1}
I20260809 23:04:07.417042 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:07.422469 16416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd6179379544aceb8e8f93d0383a2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40085 } }
I20260809 23:04:07.422909 16416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:07.423018 16416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd6179379544aceb8e8f93d0383a2bd, State: Initialized, Role: FOLLOWER
I20260809 23:04:07.423638 16416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd6179379544aceb8e8f93d0383a2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40085 } }
I20260809 23:04:07.423910 16416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:07.424049 16416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:07.424253 16416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:07.426537 16416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd6179379544aceb8e8f93d0383a2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40085 } }
I20260809 23:04:07.427040 16416 leader_election.cc:302] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dd6179379544aceb8e8f93d0383a2bd; no voters:
I20260809 23:04:07.427680 16416 leader_election.cc:288] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:07.427794 16428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:07.429320 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:07.422444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40417 (local address 127.0.0.1:40085)
0809 23:04:07.422621 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:07.422626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:07.423152 (+ 526us) server_negotiation.cc:407] Connection header received
0809 23:04:07.423392 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:07.423398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:07.423470 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:07.423565 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:07.424206 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:07.424998 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:07.426425 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:07.426985 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:07.427137 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:07.428412 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:07.428419 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:07.428423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:07.428445 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:07.428493 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:07.428516 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:07.428581 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:07.428834 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 23:04:07.428942 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 23:04:07.429631 16428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [term 1 LEADER]: Becoming Leader. State: Replica: 8dd6179379544aceb8e8f93d0383a2bd, State: Running, Role: LEADER
I20260809 23:04:07.430159 16416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:07.430225 16428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd6179379544aceb8e8f93d0383a2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40085 } }
I20260809 23:04:07.434106 16429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dd6179379544aceb8e8f93d0383a2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd6179379544aceb8e8f93d0383a2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40085 } } }
I20260809 23:04:07.434674 16429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:07.433564 16430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dd6179379544aceb8e8f93d0383a2bd. Latest consensus state: current_term: 1 leader_uuid: "8dd6179379544aceb8e8f93d0383a2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd6179379544aceb8e8f93d0383a2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40085 } } }
I20260809 23:04:07.435550 16430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:07.436393 16436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:07.441927 16436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:07.450347 16436 catalog_manager.cc:1347] Generated new cluster ID: 61063b8d67fa463a8a2cd6545dfee99c
I20260809 23:04:07.450484 16436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:07.468324 16436 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:07.470150 16436 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:07.487427 16436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dd6179379544aceb8e8f93d0383a2bd: Generated new TSK 0
I20260809 23:04:07.491878 16436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:07.509130 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16336
W20260809 23:04:07.510679 16411 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50239
W20260809 23:04:07.511566 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16336: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:07.511726 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 16336
2026-08-09T23:04:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2083 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:07 dist-test-slave-vxj2 krb5kdc[16453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:07 dist-test-slave-vxj2 krb5kdc[16453](info): set up 2 sockets
Aug 09 23:04:07 dist-test-slave-vxj2 krb5kdc[16453](info): commencing operation
krb5kdc: starting...
W20260809 23:04:08.705061 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:08 dist-test-slave-vxj2 krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:08.978165 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33083
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:09.313216 16467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:09.313489 16467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:09.313673 16467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:09.320686 16467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:09.320784 16467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:09.320894 16467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:09.320995 16467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:09.321067 16467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:09.330679 16467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:09.331894 16467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:09.333189 16467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:09.344925 16472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:09.345000 16473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:09.344972 16475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:09.345801 16467 server_base.cc:1034] running on GCE node
I20260809 23:04:09.346568 16467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:09.347827 16467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:09.349059 16467 hybrid_clock.cc:648] HybridClock initialized: now 1786316649349015 us; error 48 us; skew 500 ppm
I20260809 23:04:09.349416 16467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:09 dist-test-slave-vxj2 krb5kdc[16453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:09.352476 16467 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:09.353987 16467 webserver.cc:460] Webserver started at http://127.0.0.1:39613/ using document root <none> and password file <none>
I20260809 23:04:09.354591 16467 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:09.354725 16467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:09.355006 16467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:09.358108 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "706cfd42d6d24aaaa02947cac3c06cc5"
format_stamp: "Formatted at 2026-08-09 23:04:09 on dist-test-slave-vxj2"
server_key: "254ffb18ea178b11adf189541e2798a7"
server_key_iv: "25815e1783dd8a068bf64f3529ceaf76"
server_key_version: "encryptionkey@0"
I20260809 23:04:09.358951 16467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "706cfd42d6d24aaaa02947cac3c06cc5"
format_stamp: "Formatted at 2026-08-09 23:04:09 on dist-test-slave-vxj2"
server_key: "254ffb18ea178b11adf189541e2798a7"
server_key_iv: "25815e1783dd8a068bf64f3529ceaf76"
server_key_version: "encryptionkey@0"
I20260809 23:04:09.364106 16467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 23:04:09.367545 16485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:09.368815 16467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:04:09.369002 16467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "706cfd42d6d24aaaa02947cac3c06cc5"
format_stamp: "Formatted at 2026-08-09 23:04:09 on dist-test-slave-vxj2"
server_key: "254ffb18ea178b11adf189541e2798a7"
server_key_iv: "25815e1783dd8a068bf64f3529ceaf76"
server_key_version: "encryptionkey@0"
I20260809 23:04:09.369231 16467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:09.375123 16467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:09.379362 16467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:09.379662 16467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:09.403718 16467 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33083
I20260809 23:04:09.403810 16545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33083 every 8 connection(s)
I20260809 23:04:09.405197 16467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:09.409022 16546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:09.415216 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 16467
I20260809 23:04:09.415422 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:09.415997 24703 external_mini_cluster.cc:1428] Setting key 0f65d132c03da13b87dba37e340db28d
I20260809 23:04:09.420085 16546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5: Bootstrap starting.
I20260809 23:04:09.423712 16546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:09.423933 16549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:09.417850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49877 (local address 127.0.0.1:33083)
0809 23:04:09.418652 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:09.418663 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:09.418702 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:04:09.420540 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:09.420563 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:09.420914 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:09.421337 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:09.422414 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:09.422420 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:09.422432 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:09.422446 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:09.422506 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:09.422525 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:09.422690 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:09.422811 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 23:04:09.423013 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":374,"threads_started":1}
I20260809 23:04:09.424844 16546 log.cc:824] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:09.427829 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:09.428164 16546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5: No bootstrap required, opened a new log
I20260809 23:04:09.434770 16549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:09.432356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34841 (local address 127.0.0.1:33083)
0809 23:04:09.432519 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:09.432523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:09.432819 (+ 296us) server_negotiation.cc:407] Connection header received
0809 23:04:09.432961 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:09.432964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:09.433013 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:09.433096 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:09.434062 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:09.434066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:09.434069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:09.434089 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:09.434131 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:09.434146 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:09.434254 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:09.434386 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 23:04:09.434430 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:04:09.437117 16546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706cfd42d6d24aaaa02947cac3c06cc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33083 } }
I20260809 23:04:09.437541 16546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:09.437647 16546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 706cfd42d6d24aaaa02947cac3c06cc5, State: Initialized, Role: FOLLOWER
I20260809 23:04:09.438184 16546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706cfd42d6d24aaaa02947cac3c06cc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33083 } }
I20260809 23:04:09.438457 16546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:09.438599 16546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:09.438791 16546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:09.441313 16546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706cfd42d6d24aaaa02947cac3c06cc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33083 } }
I20260809 23:04:09.441794 16546 leader_election.cc:302] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 706cfd42d6d24aaaa02947cac3c06cc5; no voters:
I20260809 23:04:09.442459 16546 leader_election.cc:288] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:09.442584 16557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:09.443341 16557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [term 1 LEADER]: Becoming Leader. State: Replica: 706cfd42d6d24aaaa02947cac3c06cc5, State: Running, Role: LEADER
I20260809 23:04:09.443964 16557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706cfd42d6d24aaaa02947cac3c06cc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33083 } }
I20260809 23:04:09.444576 16546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:09.447069 16559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 706cfd42d6d24aaaa02947cac3c06cc5. Latest consensus state: current_term: 1 leader_uuid: "706cfd42d6d24aaaa02947cac3c06cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706cfd42d6d24aaaa02947cac3c06cc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33083 } } }
I20260809 23:04:09.447546 16559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:09.447108 16558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "706cfd42d6d24aaaa02947cac3c06cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706cfd42d6d24aaaa02947cac3c06cc5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33083 } } }
I20260809 23:04:09.448262 16558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:09.453876 16565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:09.460186 16565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:09.468900 16565 catalog_manager.cc:1347] Generated new cluster ID: 88b2a55241da47c0b3fc86ddf0ff94c7
I20260809 23:04:09.469017 16565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:09.483986 16565 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:09.484441 16565 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:09.491444 16565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 706cfd42d6d24aaaa02947cac3c06cc5: Generated new TSK 0
I20260809 23:04:09.492164 16565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:09.501559 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16467
W20260809 23:04:09.502709 16541 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49877
W20260809 23:04:09.503582 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16467: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:09.503743 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 16467
2026-08-09T23:04:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1991 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:09 dist-test-slave-vxj2 krb5kdc[16581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:09 dist-test-slave-vxj2 krb5kdc[16581](info): set up 2 sockets
Aug 09 23:04:09 dist-test-slave-vxj2 krb5kdc[16581](info): commencing operation
krb5kdc: starting...
W20260809 23:04:10.721879 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:10 dist-test-slave-vxj2 krb5kdc[16581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:11.013226 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:11.360646 16595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:11.360926 16595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:11.361163 16595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:11.368417 16595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:11.368521 16595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:11.368667 16595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:11.368775 16595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:11.368862 16595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:11.378360 16595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:11.379549 16595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:11.381054 16595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:11.395061 16602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:11.395071 16603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:11.395174 16605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:11.395825 16595 server_base.cc:1034] running on GCE node
I20260809 23:04:11.396629 16595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:11.398124 16595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:11.399371 16595 hybrid_clock.cc:648] HybridClock initialized: now 1786316651399338 us; error 58 us; skew 500 ppm
I20260809 23:04:11.399737 16595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:11 dist-test-slave-vxj2 krb5kdc[16581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316651, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:11.403033 16595 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:11.404531 16595 webserver.cc:460] Webserver started at http://127.0.0.1:38287/ using document root <none> and password file <none>
I20260809 23:04:11.405445 16595 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:11.405608 16595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:11.405933 16595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:11.408921 16595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "3be9a9f94a25447592db71317fad5f03"
format_stamp: "Formatted at 2026-08-09 23:04:11 on dist-test-slave-vxj2"
server_key: "64e9fa94eb470fa90e5d374076237f94"
server_key_iv: "e4662c3b025d819a338a5ac0d2e4bcb8"
server_key_version: "encryptionkey@0"
I20260809 23:04:11.409734 16595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "3be9a9f94a25447592db71317fad5f03"
format_stamp: "Formatted at 2026-08-09 23:04:11 on dist-test-slave-vxj2"
server_key: "64e9fa94eb470fa90e5d374076237f94"
server_key_iv: "e4662c3b025d819a338a5ac0d2e4bcb8"
server_key_version: "encryptionkey@0"
I20260809 23:04:11.414832 16595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:04:11.418160 16612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:11.419422 16595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:04:11.419610 16595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "3be9a9f94a25447592db71317fad5f03"
format_stamp: "Formatted at 2026-08-09 23:04:11 on dist-test-slave-vxj2"
server_key: "64e9fa94eb470fa90e5d374076237f94"
server_key_iv: "e4662c3b025d819a338a5ac0d2e4bcb8"
server_key_version: "encryptionkey@0"
I20260809 23:04:11.419801 16595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:11.449016 16595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:11.453250 16595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:11.453554 16595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:11.472870 16595 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36391
I20260809 23:04:11.472921 16673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36391 every 8 connection(s)
I20260809 23:04:11.474287 16595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:11.478207 16674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:11.482388 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 16595
I20260809 23:04:11.482600 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:11.483163 24703 external_mini_cluster.cc:1428] Setting key 4ec3d0bec16d258324771d6a5c0955be
I20260809 23:04:11.490939 16674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03: Bootstrap starting.
I20260809 23:04:11.494011 16674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:11.495190 16674 log.cc:824] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03: Log is configured to *not* fsync() on all Append() calls
Aug 09 23:04:11 dist-test-slave-vxj2 krb5kdc[16581](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:04:11.498163 16674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03: No bootstrap required, opened a new log
I20260809 23:04:11.506613 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:11.485103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:58299 (local address 127.0.0.1:36391)
0809 23:04:11.485867 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:11.485878 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:11.485916 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:04:11.487582 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:11.487602 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:11.488060 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:11.488616 (+ 556us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:11.490259 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:11.491717 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:11.493070 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:11.493831 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:11.494066 (+ 235us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:11.498883 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:11.498891 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:11.498912 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:11.498955 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:11.502325 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:11.503186 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:11.503192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:11.503197 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:11.503375 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:11.503872 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:11.503876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:11.503878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:11.504419 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:11.504784 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:11.505280 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 23:04:11.505601 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":313,"threads_started":1}
I20260809 23:04:11.506402 16674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9a9f94a25447592db71317fad5f03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260809 23:04:11.506846 16674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:11.507017 16674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3be9a9f94a25447592db71317fad5f03, State: Initialized, Role: FOLLOWER
I20260809 23:04:11.507675 16674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9a9f94a25447592db71317fad5f03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260809 23:04:11.508029 16674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:11.508198 16674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:11.508407 16674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:11.511163 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:11.511615 16674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9a9f94a25447592db71317fad5f03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260809 23:04:11.512117 16674 leader_election.cc:302] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3be9a9f94a25447592db71317fad5f03; no voters:
I20260809 23:04:11.512817 16674 leader_election.cc:288] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:11.512915 16680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:11.515565 16674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:11.518992 16680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [term 1 LEADER]: Becoming Leader. State: Replica: 3be9a9f94a25447592db71317fad5f03, State: Running, Role: LEADER
I20260809 23:04:11.519819 16680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9a9f94a25447592db71317fad5f03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } }
I20260809 23:04:11.523262 16681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3be9a9f94a25447592db71317fad5f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9a9f94a25447592db71317fad5f03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } } }
I20260809 23:04:11.523756 16681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:11.524401 16684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3be9a9f94a25447592db71317fad5f03. Latest consensus state: current_term: 1 leader_uuid: "3be9a9f94a25447592db71317fad5f03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3be9a9f94a25447592db71317fad5f03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36391 } } }
I20260809 23:04:11.524703 16684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:11.526417 16700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:11.532917 16700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:11.547649 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:11.522954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:46063 (local address 127.0.0.1:36391)
0809 23:04:11.525087 (+ 2133us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:11.525096 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:11.525128 (+ 32us) server_negotiation.cc:407] Connection header received
0809 23:04:11.525328 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:11.525337 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:11.525458 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:11.525579 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:11.528271 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:11.529961 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:11.532436 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:11.533436 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:11.533575 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:11.538159 (+ 4584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:11.538169 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:11.538177 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:11.538229 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:11.542896 (+ 4667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:11.543983 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:11.543990 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:11.543994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:11.544154 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:11.544911 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:11.544919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:11.544925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:11.545506 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:11.545845 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:11.546826 (+ 981us) server_negotiation.cc:299] Negotiation successful
0809 23:04:11.547088 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1950}
I20260809 23:04:11.552260 16700 catalog_manager.cc:1347] Generated new cluster ID: a82fa335a2e249ddbf9b0ad637641b67
I20260809 23:04:11.552394 16700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:11.584784 16700 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:11.586356 16700 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:11.595077 16700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3be9a9f94a25447592db71317fad5f03: Generated new TSK 0
I20260809 23:04:11.595896 16700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:11.611363 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16595
I20260809 23:04:11.612697 16669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:11.709041 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 16595
W20260809 23:04:11.729624 16579 connection.cc:535] client connection to 127.0.0.1:36391 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36391): Connection reset by peer (error 104)
2026-08-09T23:04:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2205 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:11 dist-test-slave-vxj2 krb5kdc[16716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:11 dist-test-slave-vxj2 krb5kdc[16716](info): set up 2 sockets
Aug 09 23:04:11 dist-test-slave-vxj2 krb5kdc[16716](info): commencing operation
krb5kdc: starting...
W20260809 23:04:12.899573 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.007s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:13 dist-test-slave-vxj2 krb5kdc[16716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:13.184446 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:13.526623 16730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:13.526943 16730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:13.527176 16730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:13.535542 16730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:13.535662 16730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:13.535764 16730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:13.535908 16730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:13.536001 16730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:13.546379 16730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:13.547789 16730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:13.549463 16730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:13.562424 16736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:13.562547 16735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:13.562527 16739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:13.563778 16730 server_base.cc:1034] running on GCE node
I20260809 23:04:13.564556 16730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:13.566098 16730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:13.567319 16730 hybrid_clock.cc:648] HybridClock initialized: now 1786316653567289 us; error 31 us; skew 500 ppm
I20260809 23:04:13.567633 16730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:13 dist-test-slave-vxj2 krb5kdc[16716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:13.576470 16730 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:13.578110 16730 webserver.cc:460] Webserver started at http://127.0.0.1:42553/ using document root <none> and password file <none>
I20260809 23:04:13.578765 16730 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:13.578881 16730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:13.579159 16730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:13.582219 16730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "48712c0f42f64ef3b924c4598b27d12a"
format_stamp: "Formatted at 2026-08-09 23:04:13 on dist-test-slave-vxj2"
server_key: "c19fbbcc91a5e2dbeb9cf4511b4b7a8d"
server_key_iv: "bbd5e041c99ecc0dc44f0d1486e12678"
server_key_version: "encryptionkey@0"
I20260809 23:04:13.582952 16730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "48712c0f42f64ef3b924c4598b27d12a"
format_stamp: "Formatted at 2026-08-09 23:04:13 on dist-test-slave-vxj2"
server_key: "c19fbbcc91a5e2dbeb9cf4511b4b7a8d"
server_key_iv: "bbd5e041c99ecc0dc44f0d1486e12678"
server_key_version: "encryptionkey@0"
I20260809 23:04:13.588248 16730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 23:04:13.591670 16746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:13.593086 16730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:04:13.593319 16730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "48712c0f42f64ef3b924c4598b27d12a"
format_stamp: "Formatted at 2026-08-09 23:04:13 on dist-test-slave-vxj2"
server_key: "c19fbbcc91a5e2dbeb9cf4511b4b7a8d"
server_key_iv: "bbd5e041c99ecc0dc44f0d1486e12678"
server_key_version: "encryptionkey@0"
I20260809 23:04:13.593521 16730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:13.612541 16730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:13.616983 16730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:13.617293 16730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:13.637482 16730 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35531
I20260809 23:04:13.637542 16802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35531 every 8 connection(s)
I20260809 23:04:13.638911 16730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:13.642787 16803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:13.645531 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 16730
I20260809 23:04:13.645701 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:13.646167 24703 external_mini_cluster.cc:1428] Setting key ebb591e6bb8fc8f1c1b6de7b316150a7
I20260809 23:04:13.657461 16803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a: Bootstrap starting.
I20260809 23:04:13.661751 16803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:13.662786 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:13.648082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:48645 (local address 127.0.0.1:35531)
0809 23:04:13.649244 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:13.649259 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:13.649314 (+ 55us) server_negotiation.cc:407] Connection header received
0809 23:04:13.651961 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:13.651993 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:13.652512 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:13.653015 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:13.653589 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:13.655773 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:13.657104 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:13.658266 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:13.658460 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:13.659685 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:13.659699 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:13.659729 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:13.660497 (+ 768us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:13.660614 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:13.660655 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:13.660846 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:13.661147 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 23:04:13.661347 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":563,"threads_started":1}
I20260809 23:04:13.663197 16803 log.cc:824] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:13.666363 16803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a: No bootstrap required, opened a new log
I20260809 23:04:13.667026 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:13.677251 16803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48712c0f42f64ef3b924c4598b27d12a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35531 } }
I20260809 23:04:13.677740 16803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:13.677902 16803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48712c0f42f64ef3b924c4598b27d12a, State: Initialized, Role: FOLLOWER
I20260809 23:04:13.678643 16803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48712c0f42f64ef3b924c4598b27d12a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35531 } }
I20260809 23:04:13.678930 16803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:13.679104 16803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:13.679319 16803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:13.680492 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:13.672859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:43255 (local address 127.0.0.1:35531)
0809 23:04:13.673039 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:13.673044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:13.673425 (+ 381us) server_negotiation.cc:407] Connection header received
0809 23:04:13.673648 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:13.673652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:13.673722 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:13.673808 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:13.674464 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:13.675342 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:13.677296 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:13.677859 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:13.678043 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:13.679484 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:13.679492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:13.679497 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:13.679520 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:13.679596 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:13.679620 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:13.679724 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:13.680002 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 23:04:13.680081 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 23:04:13.682149 16803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48712c0f42f64ef3b924c4598b27d12a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35531 } }
I20260809 23:04:13.682657 16803 leader_election.cc:302] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48712c0f42f64ef3b924c4598b27d12a; no voters:
I20260809 23:04:13.683230 16803 leader_election.cc:288] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:13.683370 16819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:13.684082 16819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [term 1 LEADER]: Becoming Leader. State: Replica: 48712c0f42f64ef3b924c4598b27d12a, State: Running, Role: LEADER
I20260809 23:04:13.684876 16819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48712c0f42f64ef3b924c4598b27d12a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35531 } }
I20260809 23:04:13.685559 16803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:13.687690 16822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48712c0f42f64ef3b924c4598b27d12a. Latest consensus state: current_term: 1 leader_uuid: "48712c0f42f64ef3b924c4598b27d12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48712c0f42f64ef3b924c4598b27d12a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35531 } } }
I20260809 23:04:13.688071 16822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:13.688644 16821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48712c0f42f64ef3b924c4598b27d12a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48712c0f42f64ef3b924c4598b27d12a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35531 } } }
I20260809 23:04:13.689035 16821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:13.690501 16826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:13.696139 16826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:13.705292 16826 catalog_manager.cc:1347] Generated new cluster ID: 3c8c32667ee74b04a1b953d8f5ed484c
I20260809 23:04:13.705407 16826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:13.724707 16826 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:13.726423 16826 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:13.732347 16826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48712c0f42f64ef3b924c4598b27d12a: Generated new TSK 0
I20260809 23:04:13.733237 16826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:13.740919 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16730
W20260809 23:04:13.742060 16798 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.142:48645
W20260809 23:04:13.742848 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16730: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:13.742985 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 16730
W20260809 23:04:13.768311 16714 connection.cc:535] client connection to 127.0.0.1:35531 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35531): Connection reset by peer (error 104)
2026-08-09T23:04:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2040 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:13 dist-test-slave-vxj2 krb5kdc[16845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:13 dist-test-slave-vxj2 krb5kdc[16845](info): set up 2 sockets
Aug 09 23:04:13 dist-test-slave-vxj2 krb5kdc[16845](info): commencing operation
krb5kdc: starting...
W20260809 23:04:14.943202 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:15 dist-test-slave-vxj2 krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:15.236609 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:15.579123 16860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:15.579452 16860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:15.579658 16860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:15.586699 16860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:15.586856 16860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:15.586966 16860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:15.587080 16860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:15.587168 16860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:15.596848 16860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:15.598186 16860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:15.599522 16860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:15.611616 16867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:15.611654 16866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:15.611569 16869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:15.612353 16860 server_base.cc:1034] running on GCE node
I20260809 23:04:15.613112 16860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:15.614622 16860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:15.615908 16860 hybrid_clock.cc:648] HybridClock initialized: now 1786316655615850 us; error 46 us; skew 500 ppm
I20260809 23:04:15.616376 16860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:15 dist-test-slave-vxj2 krb5kdc[16845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:15.619983 16860 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:15.621726 16860 webserver.cc:460] Webserver started at http://127.0.0.1:35809/ using document root <none> and password file <none>
I20260809 23:04:15.622488 16860 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:15.622670 16860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:15.623036 16860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:15.626931 16860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "c37b7e032c004c2381e2468c5c45c0a6"
format_stamp: "Formatted at 2026-08-09 23:04:15 on dist-test-slave-vxj2"
server_key: "7800b2e6cc9fe73427d7e493c36cfbac"
server_key_iv: "b29a534fb4833d2d21a6c78559e5bff5"
server_key_version: "encryptionkey@0"
I20260809 23:04:15.628139 16860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "c37b7e032c004c2381e2468c5c45c0a6"
format_stamp: "Formatted at 2026-08-09 23:04:15 on dist-test-slave-vxj2"
server_key: "7800b2e6cc9fe73427d7e493c36cfbac"
server_key_iv: "b29a534fb4833d2d21a6c78559e5bff5"
server_key_version: "encryptionkey@0"
I20260809 23:04:15.634410 16860 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260809 23:04:15.638331 16876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:15.639700 16860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:04:15.639952 16860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "c37b7e032c004c2381e2468c5c45c0a6"
format_stamp: "Formatted at 2026-08-09 23:04:15 on dist-test-slave-vxj2"
server_key: "7800b2e6cc9fe73427d7e493c36cfbac"
server_key_iv: "b29a534fb4833d2d21a6c78559e5bff5"
server_key_version: "encryptionkey@0"
I20260809 23:04:15.640195 16860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:15.646690 16860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:15.651777 16860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:15.652143 16860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:15.674146 16860 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35931
I20260809 23:04:15.674182 16933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35931 every 8 connection(s)
I20260809 23:04:15.675756 16860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:15.679638 16934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:15.684998 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 16860
I20260809 23:04:15.685281 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:15.685796 24703 external_mini_cluster.cc:1428] Setting key 522a98cce6b5cd1e0dfdceb9e946d186
I20260809 23:04:15.690649 16934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6: Bootstrap starting.
I20260809 23:04:15.694340 16934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:15.694816 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:15.687830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:49837 (local address 127.0.0.1:35931)
0809 23:04:15.688943 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:15.688956 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:15.689002 (+ 46us) server_negotiation.cc:407] Connection header received
0809 23:04:15.691067 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:15.691089 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:15.691428 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:15.691799 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:15.692957 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:15.692964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:15.692977 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:15.692994 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:15.693061 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:15.693088 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:15.693362 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:15.693476 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 23:04:15.693778 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":578,"threads_started":1}
I20260809 23:04:15.695458 16934 log.cc:824] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:15.698407 16934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6: No bootstrap required, opened a new log
I20260809 23:04:15.698822 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:15.706921 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:15.704259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:35221 (local address 127.0.0.1:35931)
0809 23:04:15.704446 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:15.704453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:15.704933 (+ 480us) server_negotiation.cc:407] Connection header received
0809 23:04:15.705071 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:15.705076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:15.705139 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:15.705294 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:15.706189 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:15.706194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:15.706199 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:15.706220 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:15.706289 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:15.706310 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:15.706475 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:15.706548 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 23:04:15.706613 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 23:04:15.708645 16934 raft_consensus.cc:348] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37b7e032c004c2381e2468c5c45c0a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260809 23:04:15.709002 16934 raft_consensus.cc:374] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:15.709077 16934 raft_consensus.cc:729] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37b7e032c004c2381e2468c5c45c0a6, State: Initialized, Role: FOLLOWER
I20260809 23:04:15.709666 16934 consensus_queue.cc:260] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37b7e032c004c2381e2468c5c45c0a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260809 23:04:15.709890 16934 raft_consensus.cc:388] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:15.709992 16934 raft_consensus.cc:482] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:15.710127 16934 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:15.712663 16934 raft_consensus.cc:504] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37b7e032c004c2381e2468c5c45c0a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260809 23:04:15.713228 16934 leader_election.cc:302] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c37b7e032c004c2381e2468c5c45c0a6; no voters:
I20260809 23:04:15.713786 16934 leader_election.cc:288] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:15.713974 16944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:15.714681 16944 raft_consensus.cc:686] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [term 1 LEADER]: Becoming Leader. State: Replica: c37b7e032c004c2381e2468c5c45c0a6, State: Running, Role: LEADER
I20260809 23:04:15.715251 16944 consensus_queue.cc:237] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37b7e032c004c2381e2468c5c45c0a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } }
I20260809 23:04:15.715934 16934 sys_catalog.cc:564] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:15.718164 16946 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c37b7e032c004c2381e2468c5c45c0a6. Latest consensus state: current_term: 1 leader_uuid: "c37b7e032c004c2381e2468c5c45c0a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37b7e032c004c2381e2468c5c45c0a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } } }
I20260809 23:04:15.719210 16946 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:15.719336 16945 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c37b7e032c004c2381e2468c5c45c0a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37b7e032c004c2381e2468c5c45c0a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35931 } } }
I20260809 23:04:15.719758 16945 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:15.725891 16952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:15.731937 16952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:15.741181 16952 catalog_manager.cc:1347] Generated new cluster ID: f53acffc325a40ff8de5334d13cb26ee
I20260809 23:04:15.741333 16952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:15.762075 16952 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:15.762595 16952 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:15.769680 16952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c37b7e032c004c2381e2468c5c45c0a6: Generated new TSK 0
I20260809 23:04:15.770766 16952 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:15.777518 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16860
W20260809 23:04:15.779141 16929 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.142:49837
W20260809 23:04:15.779871 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16860: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:15.780030 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 16860
2026-08-09T23:04:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2030 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:15 dist-test-slave-vxj2 krb5kdc[16968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:15 dist-test-slave-vxj2 krb5kdc[16968](info): set up 2 sockets
Aug 09 23:04:15 dist-test-slave-vxj2 krb5kdc[16968](info): commencing operation
krb5kdc: starting...
W20260809 23:04:16.965528 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:17 dist-test-slave-vxj2 krb5kdc[16968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:17.245414 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:17.580747 16984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:17.581013 16984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:17.581207 16984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:17.588322 16984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:17.588425 16984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:17.588543 16984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:17.588629 16984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:17.588717 16984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:17.598546 16984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:17.599756 16984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:17.601070 16984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:17.612958 16989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:17.613051 16990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:17.613005 16993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:17.613909 16984 server_base.cc:1034] running on GCE node
I20260809 23:04:17.614589 16984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:17.615901 16984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:17.617112 16984 hybrid_clock.cc:648] HybridClock initialized: now 1786316657617073 us; error 32 us; skew 500 ppm
I20260809 23:04:17.617604 16984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:17 dist-test-slave-vxj2 krb5kdc[16968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316657, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:17.620635 16984 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:17.622087 16984 webserver.cc:460] Webserver started at http://127.0.0.1:37459/ using document root <none> and password file <none>
I20260809 23:04:17.622679 16984 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:17.622784 16984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:17.623028 16984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:17.625958 16984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "57787d51575b4bd3ae89b01866d0a249"
format_stamp: "Formatted at 2026-08-09 23:04:17 on dist-test-slave-vxj2"
server_key: "c73006553e7bf8d5954a718c0d824378"
server_key_iv: "11c0a44d92bff5a91b043b3761e01143"
server_key_version: "encryptionkey@0"
I20260809 23:04:17.626713 16984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "57787d51575b4bd3ae89b01866d0a249"
format_stamp: "Formatted at 2026-08-09 23:04:17 on dist-test-slave-vxj2"
server_key: "c73006553e7bf8d5954a718c0d824378"
server_key_iv: "11c0a44d92bff5a91b043b3761e01143"
server_key_version: "encryptionkey@0"
I20260809 23:04:17.631757 16984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:04:17.635555 17002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:17.637048 16984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:04:17.637321 16984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "57787d51575b4bd3ae89b01866d0a249"
format_stamp: "Formatted at 2026-08-09 23:04:17 on dist-test-slave-vxj2"
server_key: "c73006553e7bf8d5954a718c0d824378"
server_key_iv: "11c0a44d92bff5a91b043b3761e01143"
server_key_version: "encryptionkey@0"
I20260809 23:04:17.637640 16984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:17.653829 16984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:17.658082 16984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:17.658399 16984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:17.676314 16984 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40669
I20260809 23:04:17.676362 17061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40669 every 8 connection(s)
I20260809 23:04:17.677690 16984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:17.681486 17062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:17.681977 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 16984
I20260809 23:04:17.682158 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:17.682622 24703 external_mini_cluster.cc:1428] Setting key ed1a2c7f1451d2ffbf605ba627a86952
I20260809 23:04:17.696096 17062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249: Bootstrap starting.
Aug 09 23:04:17 dist-test-slave-vxj2 krb5kdc[16968](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:04:17.699316 17062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:17.700659 17062 log.cc:824] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:17.703577 17062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249: No bootstrap required, opened a new log
I20260809 23:04:17.707615 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:17.684445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:37369 (local address 127.0.0.1:40669)
0809 23:04:17.685313 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:17.685329 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:17.685383 (+ 54us) server_negotiation.cc:407] Connection header received
0809 23:04:17.688278 (+ 2895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:17.688306 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:17.688811 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:17.689284 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:17.690615 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:17.692717 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:17.694115 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:17.695193 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:17.695392 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:17.699808 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:17.699822 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:17.699841 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:17.699891 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:17.703289 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:17.704113 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:17.704119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:17.704124 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:17.704305 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:17.704807 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:17.704811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:17.704813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:17.705419 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:17.705813 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:17.706300 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 23:04:17.706628 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":329,"threads_started":1}
I20260809 23:04:17.711824 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:17.712594 17062 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57787d51575b4bd3ae89b01866d0a249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } }
I20260809 23:04:17.712968 17062 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:17.713078 17062 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57787d51575b4bd3ae89b01866d0a249, State: Initialized, Role: FOLLOWER
I20260809 23:04:17.713706 17062 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57787d51575b4bd3ae89b01866d0a249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } }
I20260809 23:04:17.713974 17062 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:17.714115 17062 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:17.714298 17062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:17.717046 17062 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57787d51575b4bd3ae89b01866d0a249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } }
I20260809 23:04:17.717638 17062 leader_election.cc:302] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57787d51575b4bd3ae89b01866d0a249; no voters:
I20260809 23:04:17.718355 17062 leader_election.cc:288] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:17.718473 17072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:17.719966 17072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [term 1 LEADER]: Becoming Leader. State: Replica: 57787d51575b4bd3ae89b01866d0a249, State: Running, Role: LEADER
I20260809 23:04:17.720630 17072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57787d51575b4bd3ae89b01866d0a249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } }
I20260809 23:04:17.720911 17062 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:17.724609 17075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57787d51575b4bd3ae89b01866d0a249. Latest consensus state: current_term: 1 leader_uuid: "57787d51575b4bd3ae89b01866d0a249" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57787d51575b4bd3ae89b01866d0a249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } } }
I20260809 23:04:17.725121 17075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:17.726420 17074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57787d51575b4bd3ae89b01866d0a249" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57787d51575b4bd3ae89b01866d0a249" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40669 } } }
I20260809 23:04:17.726866 17074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:17.728760 17081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:17.736629 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:17.716600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:33483 (local address 127.0.0.1:40669)
0809 23:04:17.716782 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:17.716788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:17.717215 (+ 427us) server_negotiation.cc:407] Connection header received
0809 23:04:17.717466 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:17.717470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:17.717549 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:17.717671 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:17.719693 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:17.720755 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:17.722331 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:17.723230 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:17.723400 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:17.726970 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:17.726977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:17.726981 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:17.727036 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:17.731200 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:17.731997 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:17.732002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:17.732010 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:17.733381 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:17.734005 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:17.734012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:17.734018 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:17.734663 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:17.735859 (+ 1196us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:17.736075 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 23:04:17.736259 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 23:04:17.737090 17081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:17.745851 17081 catalog_manager.cc:1347] Generated new cluster ID: 3af378e86537405c9be5272e59af1f44
I20260809 23:04:17.746030 17081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:17.765322 17081 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:17.766633 17081 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:17.774890 17081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57787d51575b4bd3ae89b01866d0a249: Generated new TSK 0
I20260809 23:04:17.775697 17081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:17.790174 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 16984
I20260809 23:04:17.791378 17056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:17.883471 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 16984
W20260809 23:04:17.902118 16965 connection.cc:535] client connection to 127.0.0.1:40669 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40669): Connection reset by peer (error 104)
2026-08-09T23:04:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2100 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:17 dist-test-slave-vxj2 krb5kdc[17096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:17 dist-test-slave-vxj2 krb5kdc[17096](info): set up 2 sockets
Aug 09 23:04:17 dist-test-slave-vxj2 krb5kdc[17096](info): commencing operation
krb5kdc: starting...
W20260809 23:04:19.065800 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.007s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:19 dist-test-slave-vxj2 krb5kdc[17096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:19.345487 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38097
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:19.682638 17110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:19.682961 17110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:19.683135 17110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:19.690187 17110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:19.690290 17110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:19.690413 17110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:19.690512 17110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:19.690585 17110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:19.700328 17110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38097
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:19.701959 17110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:19.703344 17110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:19.715483 17116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:19.715544 17119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:19.715453 17117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:19.716252 17110 server_base.cc:1034] running on GCE node
I20260809 23:04:19.717121 17110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:19.718569 17110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:19.719789 17110 hybrid_clock.cc:648] HybridClock initialized: now 1786316659719728 us; error 41 us; skew 500 ppm
I20260809 23:04:19.720144 17110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:19 dist-test-slave-vxj2 krb5kdc[17096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316659, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:19.727768 17110 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:19.729321 17110 webserver.cc:460] Webserver started at http://127.0.0.1:44415/ using document root <none> and password file <none>
I20260809 23:04:19.729979 17110 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:19.730082 17110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:19.730345 17110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:19.733348 17110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "1106c110f8f445d9b4e60babafc79e72"
format_stamp: "Formatted at 2026-08-09 23:04:19 on dist-test-slave-vxj2"
server_key: "7125d16ff71237e857d32098e47f14f3"
server_key_iv: "8c9401d72122996be4e9fe7ca8bd2071"
server_key_version: "encryptionkey@0"
I20260809 23:04:19.734089 17110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "1106c110f8f445d9b4e60babafc79e72"
format_stamp: "Formatted at 2026-08-09 23:04:19 on dist-test-slave-vxj2"
server_key: "7125d16ff71237e857d32098e47f14f3"
server_key_iv: "8c9401d72122996be4e9fe7ca8bd2071"
server_key_version: "encryptionkey@0"
I20260809 23:04:19.738948 17110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:04:19.742219 17126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:19.743534 17110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 23:04:19.743697 17110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "1106c110f8f445d9b4e60babafc79e72"
format_stamp: "Formatted at 2026-08-09 23:04:19 on dist-test-slave-vxj2"
server_key: "7125d16ff71237e857d32098e47f14f3"
server_key_iv: "8c9401d72122996be4e9fe7ca8bd2071"
server_key_version: "encryptionkey@0"
I20260809 23:04:19.743880 17110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:19.760814 17110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:19.765244 17110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:19.765539 17110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:19.784920 17110 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38097
I20260809 23:04:19.784970 17185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38097 every 8 connection(s)
I20260809 23:04:19.786278 17110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:19.789844 17186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:19.792917 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 17110
I20260809 23:04:19.793097 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:19.793741 24703 external_mini_cluster.cc:1428] Setting key 5b0ffb45dd381dc27df90ab2ce553ed9
I20260809 23:04:19.802132 17186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72: Bootstrap starting.
I20260809 23:04:19.805763 17186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:19.806968 17186 log.cc:824] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:19.807471 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:19.795568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:46813 (local address 127.0.0.1:38097)
0809 23:04:19.796352 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:19.796363 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:19.796403 (+ 40us) server_negotiation.cc:407] Connection header received
0809 23:04:19.798154 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:19.798176 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:19.798579 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:19.798904 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:19.799517 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:19.800832 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:19.802823 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:19.803514 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:19.803656 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:19.805023 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:19.805030 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:19.805042 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:19.805624 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:19.805696 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:19.805720 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:19.805841 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:19.806188 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 23:04:19.806375 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":364,"threads_started":1}
I20260809 23:04:19.809880 17186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72: No bootstrap required, opened a new log
I20260809 23:04:19.811378 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:19.818064 17186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1106c110f8f445d9b4e60babafc79e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } }
I20260809 23:04:19.818480 17186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:19.818610 17186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1106c110f8f445d9b4e60babafc79e72, State: Initialized, Role: FOLLOWER
I20260809 23:04:19.819166 17186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1106c110f8f445d9b4e60babafc79e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } }
I20260809 23:04:19.819417 17186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:19.819561 17186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:19.819729 17186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:19.822808 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:19.815983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:50001 (local address 127.0.0.1:38097)
0809 23:04:19.816178 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:19.816186 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:19.816583 (+ 397us) server_negotiation.cc:407] Connection header received
0809 23:04:19.816818 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:19.816821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:19.816885 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:19.817001 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:19.817577 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:19.818334 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:19.820074 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:19.820560 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:19.820755 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:19.822036 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:19.822044 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:19.822054 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:19.822071 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:19.822128 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:19.822144 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:19.822267 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:19.822476 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 23:04:19.822530 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:04:19.822710 17186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1106c110f8f445d9b4e60babafc79e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } }
I20260809 23:04:19.823290 17186 leader_election.cc:302] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1106c110f8f445d9b4e60babafc79e72; no voters:
I20260809 23:04:19.823927 17186 leader_election.cc:288] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:19.824086 17197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:19.824769 17197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [term 1 LEADER]: Becoming Leader. State: Replica: 1106c110f8f445d9b4e60babafc79e72, State: Running, Role: LEADER
I20260809 23:04:19.825409 17197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1106c110f8f445d9b4e60babafc79e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } }
I20260809 23:04:19.826346 17186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:19.827356 17197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1106c110f8f445d9b4e60babafc79e72. Latest consensus state: current_term: 1 leader_uuid: "1106c110f8f445d9b4e60babafc79e72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1106c110f8f445d9b4e60babafc79e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } } }
I20260809 23:04:19.827642 17197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:19.828197 17199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1106c110f8f445d9b4e60babafc79e72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1106c110f8f445d9b4e60babafc79e72" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } } }
I20260809 23:04:19.828572 17199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:19.835695 17202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:19.841110 17202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:19.849905 17202 catalog_manager.cc:1347] Generated new cluster ID: e62a76a2f8cd45aaa58960d2db98aca6
I20260809 23:04:19.850036 17202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:19.874409 17202 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:19.875908 17202 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:19.882019 17202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1106c110f8f445d9b4e60babafc79e72: Generated new TSK 0
I20260809 23:04:19.883008 17202 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:19.888762 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17110
W20260809 23:04:19.889952 17181 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.142:46813
W20260809 23:04:19.890739 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17110: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:19.890944 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 17110
W20260809 23:04:19.911229 17094 connection.cc:535] client connection to 127.0.0.1:38097 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38097): Connection reset by peer (error 104)
2026-08-09T23:04:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2009 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:20 dist-test-slave-vxj2 krb5kdc[17220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:20 dist-test-slave-vxj2 krb5kdc[17220](info): set up 2 sockets
Aug 09 23:04:20 dist-test-slave-vxj2 krb5kdc[17220](info): commencing operation
krb5kdc: starting...
W20260809 23:04:21.087026 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:21 dist-test-slave-vxj2 krb5kdc[17220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:04:21.372552 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:21.718429 17234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:21.718825 17234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:21.719022 17234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:21.726588 17234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:21.726703 17234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:21.726835 17234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:21.726945 17234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:21.727010 17234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:21.737356 17234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:21.738612 17234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:21.740056 17234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:21.752942 17243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:21.752978 17240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:21.752942 17241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:21.753810 17234 server_base.cc:1034] running on GCE node
I20260809 23:04:21.754539 17234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:21.755827 17234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:21.757047 17234 hybrid_clock.cc:648] HybridClock initialized: now 1786316661757004 us; error 32 us; skew 500 ppm
I20260809 23:04:21.757423 17234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:21 dist-test-slave-vxj2 krb5kdc[17220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:21.760533 17234 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:04:21.762072 17234 webserver.cc:460] Webserver started at http://127.0.0.1:36839/ using document root <none> and password file <none>
I20260809 23:04:21.762730 17234 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:21.762890 17234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:21.763207 17234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:21.766203 17234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "1478fa70273148f88fb04f0e7d6368fe"
format_stamp: "Formatted at 2026-08-09 23:04:21 on dist-test-slave-vxj2"
server_key: "a17f86b33023ee392cd6eaaef269ce47"
server_key_iv: "a98b2db9676f637ba0a963fb6ae2bf27"
server_key_version: "encryptionkey@0"
I20260809 23:04:21.766990 17234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "1478fa70273148f88fb04f0e7d6368fe"
format_stamp: "Formatted at 2026-08-09 23:04:21 on dist-test-slave-vxj2"
server_key: "a17f86b33023ee392cd6eaaef269ce47"
server_key_iv: "a98b2db9676f637ba0a963fb6ae2bf27"
server_key_version: "encryptionkey@0"
I20260809 23:04:21.772212 17234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 23:04:21.775718 17254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:21.777047 17234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 23:04:21.777253 17234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "1478fa70273148f88fb04f0e7d6368fe"
format_stamp: "Formatted at 2026-08-09 23:04:21 on dist-test-slave-vxj2"
server_key: "a17f86b33023ee392cd6eaaef269ce47"
server_key_iv: "a98b2db9676f637ba0a963fb6ae2bf27"
server_key_version: "encryptionkey@0"
I20260809 23:04:21.777446 17234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:21.783272 17234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:21.787544 17234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:21.787896 17234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:21.808666 17234 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45459
I20260809 23:04:21.808750 17308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45459 every 8 connection(s)
I20260809 23:04:21.810101 17234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:21.811079 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 17234
I20260809 23:04:21.811270 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:21.811687 24703 external_mini_cluster.cc:1428] Setting key 8b55ac991a09c41306fcc084d843e46d
I20260809 23:04:21.814733 17309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:21.819346 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:21.813528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:40233 (local address 127.0.0.1:45459)
0809 23:04:21.814263 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:21.814274 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:21.814313 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:04:21.815950 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:21.815971 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:21.816267 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:21.816632 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:21.817799 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:21.817805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:21.817816 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:21.817844 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:21.817906 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:21.817923 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:21.818169 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:21.818326 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 23:04:21.818520 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":312,"threads_started":1}
I20260809 23:04:21.823004 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:21.826603 17309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe: Bootstrap starting.
I20260809 23:04:21.830055 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:21.827439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:54545 (local address 127.0.0.1:45459)
0809 23:04:21.827620 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:21.827626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:21.828125 (+ 499us) server_negotiation.cc:407] Connection header received
0809 23:04:21.828232 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:21.828239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:21.828305 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:21.828421 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:21.829390 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:21.829397 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:21.829401 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:21.829421 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:21.829471 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:21.829492 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:21.829627 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:21.829696 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 23:04:21.829756 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:04:21.830591 17309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:21.831895 17309 log.cc:824] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:21.834626 17309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe: No bootstrap required, opened a new log
I20260809 23:04:21.842252 17309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1478fa70273148f88fb04f0e7d6368fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45459 } }
I20260809 23:04:21.842634 17309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:21.842782 17309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1478fa70273148f88fb04f0e7d6368fe, State: Initialized, Role: FOLLOWER
I20260809 23:04:21.843375 17309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1478fa70273148f88fb04f0e7d6368fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45459 } }
I20260809 23:04:21.843611 17309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:21.843772 17309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:21.843950 17309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:21.846300 17309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1478fa70273148f88fb04f0e7d6368fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45459 } }
I20260809 23:04:21.846772 17309 leader_election.cc:302] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1478fa70273148f88fb04f0e7d6368fe; no voters:
I20260809 23:04:21.847364 17309 leader_election.cc:288] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:21.847467 17319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:21.848202 17319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [term 1 LEADER]: Becoming Leader. State: Replica: 1478fa70273148f88fb04f0e7d6368fe, State: Running, Role: LEADER
I20260809 23:04:21.848784 17319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1478fa70273148f88fb04f0e7d6368fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45459 } }
I20260809 23:04:21.849386 17309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:21.851394 17320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1478fa70273148f88fb04f0e7d6368fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1478fa70273148f88fb04f0e7d6368fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45459 } } }
I20260809 23:04:21.851768 17320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:21.852284 17321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1478fa70273148f88fb04f0e7d6368fe. Latest consensus state: current_term: 1 leader_uuid: "1478fa70273148f88fb04f0e7d6368fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1478fa70273148f88fb04f0e7d6368fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45459 } } }
I20260809 23:04:21.852691 17321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:21.852922 17325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:21.859820 17325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:21.869093 17325 catalog_manager.cc:1347] Generated new cluster ID: 104071ba55f74ceeaafb45fd254981f3
I20260809 23:04:21.869218 17325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:21.896488 17325 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:21.896991 17325 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:21.904860 17325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1478fa70273148f88fb04f0e7d6368fe: Generated new TSK 0
I20260809 23:04:21.905705 17325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:21.916638 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17234
W20260809 23:04:21.918023 17304 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.142:40233
W20260809 23:04:21.918849 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17234: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:21.918998 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 17234
2026-08-09T23:04:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2027 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:22 dist-test-slave-vxj2 krb5kdc[17345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:22 dist-test-slave-vxj2 krb5kdc[17345](info): set up 2 sockets
Aug 09 23:04:22 dist-test-slave-vxj2 krb5kdc[17345](info): commencing operation
krb5kdc: starting...
W20260809 23:04:23.106033 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:23 dist-test-slave-vxj2 krb5kdc[17345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:04:23.396576 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:37351
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:37351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:23.732226 17359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:23.732508 17359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:23.732623 17359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:23.739822 17359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:23.739933 17359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:23.739976 17359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:23.740054 17359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:23.740097 17359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:23.749990 17359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:37351
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:37351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:23.751200 17359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:23.752429 17359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:23.764403 17365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:23.764508 17364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:23.764360 17368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:23.765129 17359 server_base.cc:1034] running on GCE node
I20260809 23:04:23.765884 17359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:23.767185 17359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:23.768381 17359 hybrid_clock.cc:648] HybridClock initialized: now 1786316663768329 us; error 46 us; skew 500 ppm
I20260809 23:04:23.768661 17359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:23 dist-test-slave-vxj2 krb5kdc[17345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316663, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:23.771973 17359 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:04:23.773474 17359 webserver.cc:460] Webserver started at http://127.24.31.254:39271/ using document root <none> and password file <none>
I20260809 23:04:23.774122 17359 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:23.774281 17359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:23.774592 17359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:23.777540 17359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "6eb9d9b418c44c58ab54d2a80e225fb0"
format_stamp: "Formatted at 2026-08-09 23:04:23 on dist-test-slave-vxj2"
server_key: "4fd0ae3b7c03eb45fcb15ad7087c143f"
server_key_iv: "edf47d8e236bd44e419d4c4073c5e0cd"
server_key_version: "encryptionkey@0"
I20260809 23:04:23.778296 17359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "6eb9d9b418c44c58ab54d2a80e225fb0"
format_stamp: "Formatted at 2026-08-09 23:04:23 on dist-test-slave-vxj2"
server_key: "4fd0ae3b7c03eb45fcb15ad7087c143f"
server_key_iv: "edf47d8e236bd44e419d4c4073c5e0cd"
server_key_version: "encryptionkey@0"
I20260809 23:04:23.783499 17359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:04:23.786988 17377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:23.788349 17359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:04:23.788522 17359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "6eb9d9b418c44c58ab54d2a80e225fb0"
format_stamp: "Formatted at 2026-08-09 23:04:23 on dist-test-slave-vxj2"
server_key: "4fd0ae3b7c03eb45fcb15ad7087c143f"
server_key_iv: "edf47d8e236bd44e419d4c4073c5e0cd"
server_key_version: "encryptionkey@0"
I20260809 23:04:23.788733 17359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:23.832033 17359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:23.837616 17359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:23.837987 17359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:23.858469 17359 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:37351
I20260809 23:04:23.858502 17434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:37351 every 8 connection(s)
I20260809 23:04:23.859855 17359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:23.863560 17435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:23.865237 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 17359
I20260809 23:04:23.865444 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:23.865933 24703 external_mini_cluster.cc:1428] Setting key 65fa84115629c16fd69b70fd22563e15
I20260809 23:04:23.877725 17435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0: Bootstrap starting.
Aug 09 23:04:23 dist-test-slave-vxj2 krb5kdc[17345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:04:23.881532 17435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:23.882651 17435 log.cc:824] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:23.885466 17435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0: No bootstrap required, opened a new log
I20260809 23:04:23.889427 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:23.867725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52363 (local address 127.24.31.254:37351)
0809 23:04:23.868516 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:23.868527 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:23.868570 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:04:23.870212 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:23.870235 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:23.870609 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:23.870926 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:23.872734 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:23.874183 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:23.876011 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:23.876644 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:23.876811 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:23.882026 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:23.882046 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:23.882059 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:23.882093 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:23.885242 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:23.886067 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:23.886082 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:23.886087 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:23.886249 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:23.886753 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:23.886757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:23.886759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:23.887263 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:23.887634 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:23.888128 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 23:04:23.888431 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":374,"threads_started":1}
I20260809 23:04:23.893462 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:23.893748 17435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb9d9b418c44c58ab54d2a80e225fb0" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37351 } }
I20260809 23:04:23.894117 17435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:23.894222 17435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb9d9b418c44c58ab54d2a80e225fb0, State: Initialized, Role: FOLLOWER
I20260809 23:04:23.894740 17435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb9d9b418c44c58ab54d2a80e225fb0" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37351 } }
I20260809 23:04:23.895015 17435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:23.895156 17435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:23.895344 17435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:23.897920 17435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb9d9b418c44c58ab54d2a80e225fb0" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37351 } }
I20260809 23:04:23.898571 17435 leader_election.cc:302] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6eb9d9b418c44c58ab54d2a80e225fb0; no voters:
I20260809 23:04:23.899322 17435 leader_election.cc:288] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:23.899441 17447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:23.902094 17435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:23.907191 17447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [term 1 LEADER]: Becoming Leader. State: Replica: 6eb9d9b418c44c58ab54d2a80e225fb0, State: Running, Role: LEADER
I20260809 23:04:23.908034 17447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb9d9b418c44c58ab54d2a80e225fb0" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37351 } }
I20260809 23:04:23.911377 17448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6eb9d9b418c44c58ab54d2a80e225fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb9d9b418c44c58ab54d2a80e225fb0" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37351 } } }
I20260809 23:04:23.911837 17448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:23.912451 17449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eb9d9b418c44c58ab54d2a80e225fb0. Latest consensus state: current_term: 1 leader_uuid: "6eb9d9b418c44c58ab54d2a80e225fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eb9d9b418c44c58ab54d2a80e225fb0" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 37351 } } }
I20260809 23:04:23.912781 17449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:23.916790 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:23.898422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58949 (local address 127.24.31.254:37351)
0809 23:04:23.898618 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:23.898625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:23.899055 (+ 430us) server_negotiation.cc:407] Connection header received
0809 23:04:23.899263 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:23.899274 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:23.899345 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:23.899455 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:23.901271 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:23.902147 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:23.903444 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:23.904216 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:23.904382 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:23.907143 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:23.907151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:23.907160 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:23.907207 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:23.911178 (+ 3971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:23.912259 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:23.912267 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:23.912272 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:23.912452 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:23.913248 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:23.913256 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:23.913262 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:23.913938 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:23.915698 (+ 1760us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:23.916012 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 23:04:23.916247 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:04:23.920089 17459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:23.924677 17459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:23.933730 17459 catalog_manager.cc:1347] Generated new cluster ID: 2917a2d0d08a4b158c106d3fcd984e6e
I20260809 23:04:23.933885 17459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:23.947099 17459 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:23.948695 17459 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:23.959380 17459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6eb9d9b418c44c58ab54d2a80e225fb0: Generated new TSK 0
I20260809 23:04:23.960207 17459 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:23.970484 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17359
I20260809 23:04:23.971653 17429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:24.063639 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 17359
W20260809 23:04:24.084375 17343 connection.cc:535] client connection to 127.24.31.254:37351 recv error: Network error: failed to read from TLS socket (remote: 127.24.31.254:37351): Connection reset by peer (error 104)
2026-08-09T23:04:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2145 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:24 dist-test-slave-vxj2 krb5kdc[17472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:24 dist-test-slave-vxj2 krb5kdc[17472](info): set up 2 sockets
Aug 09 23:04:24 dist-test-slave-vxj2 krb5kdc[17472](info): commencing operation
krb5kdc: starting...
W20260809 23:04:25.264060 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:25 dist-test-slave-vxj2 krb5kdc[17472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:04:25.560956 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:35615
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:35615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:25.897390 17487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:25.897665 17487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:25.897756 17487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:25.905054 17487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:25.905217 17487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:25.905287 17487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:25.905354 17487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:25.905395 17487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:25.915350 17487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:35615
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:35615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:25.916596 17487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:25.918000 17487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:25.929751 17495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:25.929845 17492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:25.929801 17493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:25.930675 17487 server_base.cc:1034] running on GCE node
I20260809 23:04:25.931421 17487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:25.932766 17487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:25.934006 17487 hybrid_clock.cc:648] HybridClock initialized: now 1786316665933969 us; error 49 us; skew 500 ppm
I20260809 23:04:25.934371 17487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:25 dist-test-slave-vxj2 krb5kdc[17472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316665, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:25.946926 17487 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:04:25.948559 17487 webserver.cc:460] Webserver started at http://127.24.31.254:37539/ using document root <none> and password file <none>
I20260809 23:04:25.949304 17487 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:25.949461 17487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:25.949776 17487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:25.952806 17487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "c8e0feac145b4dd3a7b6464b56c19bcb"
format_stamp: "Formatted at 2026-08-09 23:04:25 on dist-test-slave-vxj2"
server_key: "afdb4c621a0fe656cdfdcb501fcbf853"
server_key_iv: "46a0437c206c34ec67c0b86fafd5f2e8"
server_key_version: "encryptionkey@0"
I20260809 23:04:25.953600 17487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "c8e0feac145b4dd3a7b6464b56c19bcb"
format_stamp: "Formatted at 2026-08-09 23:04:25 on dist-test-slave-vxj2"
server_key: "afdb4c621a0fe656cdfdcb501fcbf853"
server_key_iv: "46a0437c206c34ec67c0b86fafd5f2e8"
server_key_version: "encryptionkey@0"
I20260809 23:04:25.958698 17487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 23:04:25.962248 17504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:25.963575 17487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 23:04:25.963774 17487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "c8e0feac145b4dd3a7b6464b56c19bcb"
format_stamp: "Formatted at 2026-08-09 23:04:25 on dist-test-slave-vxj2"
server_key: "afdb4c621a0fe656cdfdcb501fcbf853"
server_key_iv: "46a0437c206c34ec67c0b86fafd5f2e8"
server_key_version: "encryptionkey@0"
I20260809 23:04:25.963991 17487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:25.988754 17487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:25.993088 17487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:25.993436 17487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:26.012852 17487 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:35615
I20260809 23:04:26.012856 17560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:35615 every 8 connection(s)
I20260809 23:04:26.014389 17487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:26.018121 17561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:26.019145 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 17487
I20260809 23:04:26.019363 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:26.019935 24703 external_mini_cluster.cc:1428] Setting key 85f166483025cc7ce7d7e17a35e1d279
I20260809 23:04:26.031859 17561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb: Bootstrap starting.
I20260809 23:04:26.033484 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:26.021847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56969 (local address 127.24.31.254:35615)
0809 23:04:26.022647 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:26.022658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:26.022708 (+ 50us) server_negotiation.cc:407] Connection header received
0809 23:04:26.024431 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:26.024454 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:26.024852 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:26.025235 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:26.025827 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:26.027071 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:26.028964 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:26.029734 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:26.029929 (+ 195us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:26.031200 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:26.031208 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:26.031220 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:26.031740 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:26.031851 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:26.031880 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:26.032051 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:26.032346 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 23:04:26.032505 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":376,"threads_started":1}
I20260809 23:04:26.035293 17561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:26.036621 17561 log.cc:824] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:26.037619 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:26.039393 17561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb: No bootstrap required, opened a new log
I20260809 23:04:26.049052 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:26.042058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37109 (local address 127.24.31.254:35615)
0809 23:04:26.042241 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:26.042248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:26.042763 (+ 515us) server_negotiation.cc:407] Connection header received
0809 23:04:26.042916 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:26.042921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:26.042992 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:26.043109 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:26.043645 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:26.044828 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:26.046216 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:26.046982 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:26.047110 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:26.048202 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:26.048208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:26.048212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:26.048234 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:26.048305 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:26.048326 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:26.048440 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:26.048675 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 23:04:26.048731 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 23:04:26.048687 17561 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e0feac145b4dd3a7b6464b56c19bcb" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35615 } }
I20260809 23:04:26.049252 17561 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:26.049363 17561 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8e0feac145b4dd3a7b6464b56c19bcb, State: Initialized, Role: FOLLOWER
I20260809 23:04:26.049896 17561 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e0feac145b4dd3a7b6464b56c19bcb" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35615 } }
I20260809 23:04:26.050175 17561 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:26.050328 17561 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:26.050513 17561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:26.052887 17561 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e0feac145b4dd3a7b6464b56c19bcb" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35615 } }
I20260809 23:04:26.053486 17561 leader_election.cc:302] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8e0feac145b4dd3a7b6464b56c19bcb; no voters:
I20260809 23:04:26.054198 17561 leader_election.cc:288] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:26.054317 17572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:26.055099 17572 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [term 1 LEADER]: Becoming Leader. State: Replica: c8e0feac145b4dd3a7b6464b56c19bcb, State: Running, Role: LEADER
I20260809 23:04:26.055688 17572 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e0feac145b4dd3a7b6464b56c19bcb" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35615 } }
I20260809 23:04:26.056407 17561 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:26.058594 17575 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8e0feac145b4dd3a7b6464b56c19bcb. Latest consensus state: current_term: 1 leader_uuid: "c8e0feac145b4dd3a7b6464b56c19bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e0feac145b4dd3a7b6464b56c19bcb" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35615 } } }
I20260809 23:04:26.058996 17575 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:26.059752 17574 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8e0feac145b4dd3a7b6464b56c19bcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8e0feac145b4dd3a7b6464b56c19bcb" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 35615 } } }
I20260809 23:04:26.060140 17574 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:26.063840 17582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:26.069237 17582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:26.078157 17582 catalog_manager.cc:1347] Generated new cluster ID: b496c9ae463b4e4e97dffef6aea02f54
I20260809 23:04:26.078286 17582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:26.102771 17582 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:26.104293 17582 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:26.116791 17582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8e0feac145b4dd3a7b6464b56c19bcb: Generated new TSK 0
I20260809 23:04:26.117796 17582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:26.128414 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17487
W20260809 23:04:26.129736 17556 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56969
W20260809 23:04:26.130561 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17487: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:26.130707 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 17487
W20260809 23:04:26.151340 17469 connection.cc:535] client connection to 127.24.31.254:35615 recv error: Network error: failed to read from TLS socket (remote: 127.24.31.254:35615): Connection reset by peer (error 104)
2026-08-09T23:04:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2067 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:26 dist-test-slave-vxj2 krb5kdc[17596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:26 dist-test-slave-vxj2 krb5kdc[17596](info): set up 2 sockets
Aug 09 23:04:26 dist-test-slave-vxj2 krb5kdc[17596](info): commencing operation
krb5kdc: starting...
W20260809 23:04:27.343300 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:27 dist-test-slave-vxj2 krb5kdc[17596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:04:27.648196 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42493
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:34071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:42493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:27.990128 17609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:27.990424 17609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:27.990631 17609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:27.997910 17609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:27.998039 17609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:27.998132 17609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:27.998234 17609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:27.998327 17609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:28.008379 17609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:34071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:42493
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:42493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:28.009738 17609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:28.011086 17609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:28.023286 17615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:28.023325 17614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:28.023942 17617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:28.023969 17609 server_base.cc:1034] running on GCE node
I20260809 23:04:28.024847 17609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:28.026207 17609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:28.027415 17609 hybrid_clock.cc:648] HybridClock initialized: now 1786316668027372 us; error 35 us; skew 500 ppm
I20260809 23:04:28.027755 17609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:28 dist-test-slave-vxj2 krb5kdc[17596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316668, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:28.031133 17609 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:04:28.032645 17609 webserver.cc:460] Webserver started at http://127.24.31.254:35901/ using document root <none> and password file <none>
I20260809 23:04:28.033391 17609 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:28.033535 17609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:28.033836 17609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:28.036916 17609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "d644480008974faca9be2595a690921b"
format_stamp: "Formatted at 2026-08-09 23:04:28 on dist-test-slave-vxj2"
server_key: "f82e5e5f3d1ceb6b7f593a9113bdcb77"
server_key_iv: "ee9a3d321759cf031a5a6c41df336de8"
server_key_version: "encryptionkey@0"
I20260809 23:04:28.037736 17609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "d644480008974faca9be2595a690921b"
format_stamp: "Formatted at 2026-08-09 23:04:28 on dist-test-slave-vxj2"
server_key: "f82e5e5f3d1ceb6b7f593a9113bdcb77"
server_key_iv: "ee9a3d321759cf031a5a6c41df336de8"
server_key_version: "encryptionkey@0"
I20260809 23:04:28.042860 17609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 23:04:28.046271 17624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:28.047581 17609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 23:04:28.047751 17609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "d644480008974faca9be2595a690921b"
format_stamp: "Formatted at 2026-08-09 23:04:28 on dist-test-slave-vxj2"
server_key: "f82e5e5f3d1ceb6b7f593a9113bdcb77"
server_key_iv: "ee9a3d321759cf031a5a6c41df336de8"
server_key_version: "encryptionkey@0"
I20260809 23:04:28.047952 17609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:28.053561 17609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:28.057554 17609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:28.057847 17609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:28.076829 17609 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:42493
I20260809 23:04:28.076907 17684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:42493 every 8 connection(s)
I20260809 23:04:28.078217 17609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:28.081995 17685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:28.088157 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 17609
I20260809 23:04:28.088357 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:28.088941 24703 external_mini_cluster.cc:1428] Setting key d20474751736c141557310bb3997e15d
I20260809 23:04:28.092762 17685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b: Bootstrap starting.
I20260809 23:04:28.096361 17685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:28.096761 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:28.090765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58157 (local address 127.24.31.254:42493)
0809 23:04:28.091590 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:28.091601 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:28.091640 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:04:28.093430 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:28.093452 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:28.093767 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:28.094109 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:28.095205 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:28.095211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:28.095223 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:28.095237 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:28.095296 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:28.095316 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:28.095477 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:28.095595 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 23:04:28.095768 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":387,"threads_started":1}
I20260809 23:04:28.097580 17685 log.cc:824] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:28.100515 17685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b: No bootstrap required, opened a new log
I20260809 23:04:28.100615 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:28.108505 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:28.105726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52103 (local address 127.24.31.254:42493)
0809 23:04:28.105930 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:28.105937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:28.106458 (+ 521us) server_negotiation.cc:407] Connection header received
0809 23:04:28.106607 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:28.106612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:28.106677 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:28.106799 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:28.107799 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:28.107810 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:28.107815 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:28.107834 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:28.107885 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:28.107909 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:28.108034 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:28.108146 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 23:04:28.108201 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 23:04:28.110574 17685 raft_consensus.cc:348] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d644480008974faca9be2595a690921b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42493 } }
I20260809 23:04:28.110948 17685 raft_consensus.cc:374] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:28.111047 17685 raft_consensus.cc:729] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d644480008974faca9be2595a690921b, State: Initialized, Role: FOLLOWER
I20260809 23:04:28.111707 17685 consensus_queue.cc:260] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d644480008974faca9be2595a690921b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42493 } }
I20260809 23:04:28.111990 17685 raft_consensus.cc:388] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:28.112149 17685 raft_consensus.cc:482] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:28.112336 17685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:28.114638 17685 raft_consensus.cc:504] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d644480008974faca9be2595a690921b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42493 } }
I20260809 23:04:28.115121 17685 leader_election.cc:302] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d644480008974faca9be2595a690921b; no voters:
I20260809 23:04:28.115761 17685 leader_election.cc:288] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:28.115934 17696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:28.116864 17696 raft_consensus.cc:686] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [term 1 LEADER]: Becoming Leader. State: Replica: d644480008974faca9be2595a690921b, State: Running, Role: LEADER
I20260809 23:04:28.117408 17696 consensus_queue.cc:237] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d644480008974faca9be2595a690921b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42493 } }
I20260809 23:04:28.117990 17685 sys_catalog.cc:564] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:28.120249 17697 sys_catalog.cc:455] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d644480008974faca9be2595a690921b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d644480008974faca9be2595a690921b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42493 } } }
I20260809 23:04:28.120327 17698 sys_catalog.cc:455] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d644480008974faca9be2595a690921b. Latest consensus state: current_term: 1 leader_uuid: "d644480008974faca9be2595a690921b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d644480008974faca9be2595a690921b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 42493 } } }
I20260809 23:04:28.120726 17697 sys_catalog.cc:458] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:28.120726 17698 sys_catalog.cc:458] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:28.122737 17704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:28.127774 17704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:28.137513 17704 catalog_manager.cc:1347] Generated new cluster ID: 67cf348b8810445b93f57356ae73770f
I20260809 23:04:28.137616 17704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:28.153705 17704 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:28.154167 17704 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:28.164706 17704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d644480008974faca9be2595a690921b: Generated new TSK 0
I20260809 23:04:28.165635 17704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:28.172883 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17609
W20260809 23:04:28.174629 17680 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58157
W20260809 23:04:28.175372 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17609: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:28.175541 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 17609
2026-08-09T23:04:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2045 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:28 dist-test-slave-vxj2 krb5kdc[17720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:28 dist-test-slave-vxj2 krb5kdc[17720](info): set up 2 sockets
Aug 09 23:04:28 dist-test-slave-vxj2 krb5kdc[17720](info): commencing operation
krb5kdc: starting...
W20260809 23:04:29.367132 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.007s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:29 dist-test-slave-vxj2 krb5kdc[17720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:04:29.668991 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:38855
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:38855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:30.011569 17736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:30.011951 17736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:30.012120 17736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:30.019603 17736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:30.019760 17736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:30.019876 17736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:30.019995 17736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:30.020077 17736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:30.030131 17736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:38855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:38855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:30.031523 17736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:30.032899 17736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:30.044994 17741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:30.045085 17742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:30.044966 17744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:30.045866 17736 server_base.cc:1034] running on GCE node
I20260809 23:04:30.046648 17736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:30.047999 17736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:30.049252 17736 hybrid_clock.cc:648] HybridClock initialized: now 1786316670049195 us; error 37 us; skew 500 ppm
I20260809 23:04:30.049613 17736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:30 dist-test-slave-vxj2 krb5kdc[17720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316670, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:30.052896 17736 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:04:30.054520 17736 webserver.cc:460] Webserver started at http://127.24.31.254:40173/ using document root <none> and password file <none>
I20260809 23:04:30.055250 17736 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:30.055408 17736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:30.055738 17736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:30.058887 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "9796f3bf3f3f49d0a6f64592724be3cf"
format_stamp: "Formatted at 2026-08-09 23:04:30 on dist-test-slave-vxj2"
server_key: "e086caa3d112d5319ee23521b0c91655"
server_key_iv: "94f668204f5a8d51c6c067f2668f4b57"
server_key_version: "encryptionkey@0"
I20260809 23:04:30.059690 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "9796f3bf3f3f49d0a6f64592724be3cf"
format_stamp: "Formatted at 2026-08-09 23:04:30 on dist-test-slave-vxj2"
server_key: "e086caa3d112d5319ee23521b0c91655"
server_key_iv: "94f668204f5a8d51c6c067f2668f4b57"
server_key_version: "encryptionkey@0"
I20260809 23:04:30.064983 17736 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 23:04:30.068599 17751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:30.070029 17736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:04:30.070228 17736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "9796f3bf3f3f49d0a6f64592724be3cf"
format_stamp: "Formatted at 2026-08-09 23:04:30 on dist-test-slave-vxj2"
server_key: "e086caa3d112d5319ee23521b0c91655"
server_key_iv: "94f668204f5a8d51c6c067f2668f4b57"
server_key_version: "encryptionkey@0"
I20260809 23:04:30.070447 17736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:30.103346 17736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:30.107941 17736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:30.108258 17736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:30.127601 17736 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:38855
I20260809 23:04:30.127636 17814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:38855 every 8 connection(s)
I20260809 23:04:30.129040 17736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:30.129467 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 17736
I20260809 23:04:30.129652 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:30.130260 24703 external_mini_cluster.cc:1428] Setting key caace089fb38ff1bb4c81f0b9ae33c7f
I20260809 23:04:30.133685 17815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 23:04:30 dist-test-slave-vxj2 krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:04:30.147284 17815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf: Bootstrap starting.
I20260809 23:04:30.151482 17815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:30.152884 17815 log.cc:824] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:30.155567 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:30.132414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60559 (local address 127.24.31.254:38855)
0809 23:04:30.133267 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:30.133284 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:30.133327 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:04:30.134997 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:30.135020 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:30.135398 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:30.135708 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:30.137600 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:30.138997 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:30.140847 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:30.141588 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:30.141717 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:30.146592 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:30.146602 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:30.146620 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:30.146668 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:30.150680 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:30.151600 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:30.151608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:30.151616 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:30.151852 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:30.152266 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:30.152271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:30.152274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:30.153002 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:30.153316 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:30.153812 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 23:04:30.154241 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":357,"threads_started":1}
I20260809 23:04:30.156463 17815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf: No bootstrap required, opened a new log
I20260809 23:04:30.159706 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:30.164726 17815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9796f3bf3f3f49d0a6f64592724be3cf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38855 } }
I20260809 23:04:30.165220 17815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:30.165352 17815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9796f3bf3f3f49d0a6f64592724be3cf, State: Initialized, Role: FOLLOWER
I20260809 23:04:30.166033 17815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9796f3bf3f3f49d0a6f64592724be3cf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38855 } }
I20260809 23:04:30.166319 17815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:30.166500 17815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:30.166678 17815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:30.169589 17815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9796f3bf3f3f49d0a6f64592724be3cf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38855 } }
I20260809 23:04:30.170169 17815 leader_election.cc:302] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9796f3bf3f3f49d0a6f64592724be3cf; no voters:
I20260809 23:04:30.170861 17815 leader_election.cc:288] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:30.171018 17825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:30.173187 17815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:30.174499 17825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [term 1 LEADER]: Becoming Leader. State: Replica: 9796f3bf3f3f49d0a6f64592724be3cf, State: Running, Role: LEADER
I20260809 23:04:30.175124 17825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9796f3bf3f3f49d0a6f64592724be3cf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38855 } }
I20260809 23:04:30.178154 17827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9796f3bf3f3f49d0a6f64592724be3cf. Latest consensus state: current_term: 1 leader_uuid: "9796f3bf3f3f49d0a6f64592724be3cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9796f3bf3f3f49d0a6f64592724be3cf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38855 } } }
I20260809 23:04:30.178252 17826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9796f3bf3f3f49d0a6f64592724be3cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9796f3bf3f3f49d0a6f64592724be3cf" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 38855 } } }
I20260809 23:04:30.178524 17827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:30.178581 17826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:30.179481 17831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:30.184060 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:30.164341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42087 (local address 127.24.31.254:38855)
0809 23:04:30.164523 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:30.164529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:30.164972 (+ 443us) server_negotiation.cc:407] Connection header received
0809 23:04:30.165095 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:30.165100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:30.165225 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:30.165345 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:30.167210 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:30.168413 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:30.169779 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:30.170450 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:30.170591 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:30.174271 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:30.174280 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:30.174288 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:30.174341 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:30.179294 (+ 4953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:30.180153 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:30.180159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:30.180162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:30.180418 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:30.180952 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:30.180959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:30.180965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:30.181488 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:30.183037 (+ 1549us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:30.183225 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 23:04:30.183530 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 23:04:30.191089 17831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:30.200578 17831 catalog_manager.cc:1347] Generated new cluster ID: af56938344274267a5ad6e1c0ccbfed9
I20260809 23:04:30.200706 17831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:30.211054 17831 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:30.212323 17831 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:30.219239 17831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9796f3bf3f3f49d0a6f64592724be3cf: Generated new TSK 0
I20260809 23:04:30.220203 17831 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:30.233789 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17736
I20260809 23:04:30.235199 17809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:30.325389 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 17736
2026-08-09T23:04:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2147 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:30 dist-test-slave-vxj2 krb5kdc[17850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:30 dist-test-slave-vxj2 krb5kdc[17850](info): set up 2 sockets
Aug 09 23:04:30 dist-test-slave-vxj2 krb5kdc[17850](info): commencing operation
krb5kdc: starting...
W20260809 23:04:31.513885 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:31 dist-test-slave-vxj2 krb5kdc[17850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:04:31.802819 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:33235
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:43535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:33235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:32.138185 17863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:32.138525 17863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:32.138695 17863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:32.145907 17863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:32.146044 17863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:32.146147 17863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:32.146256 17863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:32.146339 17863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:32.156086 17863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:43535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:33235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:33235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:32.157409 17863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:32.158747 17863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:32.170797 17869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:32.170855 17870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:32.171053 17872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:32.171799 17863 server_base.cc:1034] running on GCE node
I20260809 23:04:32.172657 17863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:32.174168 17863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:32.175414 17863 hybrid_clock.cc:648] HybridClock initialized: now 1786316672175371 us; error 42 us; skew 500 ppm
I20260809 23:04:32.175774 17863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:32 dist-test-slave-vxj2 krb5kdc[17850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316672, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:32.184947 17863 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:04:32.186756 17863 webserver.cc:460] Webserver started at http://127.24.31.254:45893/ using document root <none> and password file <none>
I20260809 23:04:32.187507 17863 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:32.187664 17863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:32.187979 17863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:32.191346 17863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "748860d64f5b45b7ba42d1e3f8df0787"
format_stamp: "Formatted at 2026-08-09 23:04:32 on dist-test-slave-vxj2"
server_key: "296750fc315fb6d276810da475836e97"
server_key_iv: "e1171b099618770c4cf8454ca5744a55"
server_key_version: "encryptionkey@0"
I20260809 23:04:32.192127 17863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "748860d64f5b45b7ba42d1e3f8df0787"
format_stamp: "Formatted at 2026-08-09 23:04:32 on dist-test-slave-vxj2"
server_key: "296750fc315fb6d276810da475836e97"
server_key_iv: "e1171b099618770c4cf8454ca5744a55"
server_key_version: "encryptionkey@0"
I20260809 23:04:32.197615 17863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 23:04:32.201467 17880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:32.202944 17863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:04:32.203138 17863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "748860d64f5b45b7ba42d1e3f8df0787"
format_stamp: "Formatted at 2026-08-09 23:04:32 on dist-test-slave-vxj2"
server_key: "296750fc315fb6d276810da475836e97"
server_key_iv: "e1171b099618770c4cf8454ca5744a55"
server_key_version: "encryptionkey@0"
I20260809 23:04:32.203356 17863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:32.216414 17863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:32.220649 17863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:32.220952 17863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:32.241487 17863 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:33235
I20260809 23:04:32.241529 17935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:33235 every 8 connection(s)
I20260809 23:04:32.242908 17863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:32.246981 17936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:32.252722 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 17863
I20260809 23:04:32.252974 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:32.253590 24703 external_mini_cluster.cc:1428] Setting key 034d7ad61b759cf85cab278e5fa944bd
I20260809 23:04:32.258654 17936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787: Bootstrap starting.
I20260809 23:04:32.262398 17936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:32.263615 17936 log.cc:824] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:32.267022 17936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787: No bootstrap required, opened a new log
I20260809 23:04:32.267872 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:32.255513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41953 (local address 127.24.31.254:33235)
0809 23:04:32.256423 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:32.256444 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:32.256494 (+ 50us) server_negotiation.cc:407] Connection header received
0809 23:04:32.258278 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:32.258303 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:32.258690 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:32.259066 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:32.259665 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:32.260987 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:32.262931 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:32.263635 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:32.263829 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:32.265435 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:32.265443 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:32.265457 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:32.266011 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:32.266091 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:32.266117 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:32.266219 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:32.266473 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 23:04:32.266762 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":448,"threads_started":1}
I20260809 23:04:32.271898 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:32.276773 17936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748860d64f5b45b7ba42d1e3f8df0787" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33235 } }
I20260809 23:04:32.277314 17936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:32.277444 17936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 748860d64f5b45b7ba42d1e3f8df0787, State: Initialized, Role: FOLLOWER
I20260809 23:04:32.278172 17936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748860d64f5b45b7ba42d1e3f8df0787" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33235 } }
I20260809 23:04:32.278488 17936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:32.278647 17936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:32.278838 17936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:32.281865 17936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748860d64f5b45b7ba42d1e3f8df0787" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33235 } }
I20260809 23:04:32.282498 17936 leader_election.cc:302] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 748860d64f5b45b7ba42d1e3f8df0787; no voters:
I20260809 23:04:32.283193 17936 leader_election.cc:288] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:32.283406 17946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:32.285074 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:32.277287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47763 (local address 127.24.31.254:33235)
0809 23:04:32.277511 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:32.277519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:32.278121 (+ 602us) server_negotiation.cc:407] Connection header received
0809 23:04:32.278282 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:32.278287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:32.278365 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:32.278487 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:32.279219 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:32.280349 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:32.281842 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:32.282526 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:32.282670 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:32.284089 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:32.284098 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:32.284106 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:32.284135 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:32.284198 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:32.284226 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:32.284304 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:32.284615 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 23:04:32.284710 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 23:04:32.285459 17946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [term 1 LEADER]: Becoming Leader. State: Replica: 748860d64f5b45b7ba42d1e3f8df0787, State: Running, Role: LEADER
I20260809 23:04:32.285614 17936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:32.286790 17946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748860d64f5b45b7ba42d1e3f8df0787" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33235 } }
I20260809 23:04:32.289798 17951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 748860d64f5b45b7ba42d1e3f8df0787. Latest consensus state: current_term: 1 leader_uuid: "748860d64f5b45b7ba42d1e3f8df0787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748860d64f5b45b7ba42d1e3f8df0787" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33235 } } }
I20260809 23:04:32.289988 17947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "748860d64f5b45b7ba42d1e3f8df0787" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "748860d64f5b45b7ba42d1e3f8df0787" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 33235 } } }
I20260809 23:04:32.290349 17951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:32.290350 17947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:32.291381 17960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:32.296118 17960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:32.304615 17960 catalog_manager.cc:1347] Generated new cluster ID: 7697c0aa695e4dadb7f53f598374fadb
I20260809 23:04:32.304738 17960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:32.328426 17960 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:32.329909 17960 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:32.337486 17960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 748860d64f5b45b7ba42d1e3f8df0787: Generated new TSK 0
I20260809 23:04:32.338573 17960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:32.351852 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17863
W20260809 23:04:32.353075 17931 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41953
W20260809 23:04:32.353893 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17863: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:32.354048 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 17863
2026-08-09T23:04:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2028 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:32 dist-test-slave-vxj2 krb5kdc[17972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:32 dist-test-slave-vxj2 krb5kdc[17972](info): set up 2 sockets
Aug 09 23:04:32 dist-test-slave-vxj2 krb5kdc[17972](info): commencing operation
krb5kdc: starting...
W20260809 23:04:33.547287 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:33 dist-test-slave-vxj2 krb5kdc[17972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:04:33.853981 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:32951
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:39049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:32951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:34.202795 17988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:34.203064 17988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:34.203169 17988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:34.210216 17988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:34.210328 17988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:34.210376 17988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:34.210428 17988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:34.210472 17988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:34.219929 17988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:39049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:32951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:32951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:34.221208 17988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:34.222510 17988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:34.234851 17995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:34.234946 17998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:34.234900 17994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:34.236367 17988 server_base.cc:1034] running on GCE node
I20260809 23:04:34.237246 17988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:34.238787 17988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:34.240005 17988 hybrid_clock.cc:648] HybridClock initialized: now 1786316674239988 us; error 52 us; skew 500 ppm
I20260809 23:04:34.240373 17988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:34 dist-test-slave-vxj2 krb5kdc[17972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316674, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:34.243677 17988 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:04:34.245232 17988 webserver.cc:460] Webserver started at http://127.24.31.254:44773/ using document root <none> and password file <none>
I20260809 23:04:34.245916 17988 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:34.246066 17988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:34.246372 17988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:34.249390 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "5b8298df64d644bd9e3f4cc770bd955b"
format_stamp: "Formatted at 2026-08-09 23:04:34 on dist-test-slave-vxj2"
server_key: "dc39bb7ab5c5fcefd061693affcea397"
server_key_iv: "bed2d2e46828f1d3f460b92ed799bad5"
server_key_version: "encryptionkey@0"
I20260809 23:04:34.250188 17988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "5b8298df64d644bd9e3f4cc770bd955b"
format_stamp: "Formatted at 2026-08-09 23:04:34 on dist-test-slave-vxj2"
server_key: "dc39bb7ab5c5fcefd061693affcea397"
server_key_iv: "bed2d2e46828f1d3f460b92ed799bad5"
server_key_version: "encryptionkey@0"
I20260809 23:04:34.255393 17988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 23:04:34.258922 18008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:34.260208 17988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 23:04:34.260391 17988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "5b8298df64d644bd9e3f4cc770bd955b"
format_stamp: "Formatted at 2026-08-09 23:04:34 on dist-test-slave-vxj2"
server_key: "dc39bb7ab5c5fcefd061693affcea397"
server_key_iv: "bed2d2e46828f1d3f460b92ed799bad5"
server_key_version: "encryptionkey@0"
I20260809 23:04:34.260593 17988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:34.266299 17988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:34.270514 17988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:34.270815 17988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:34.289856 17988 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:32951
I20260809 23:04:34.289942 18060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:32951 every 8 connection(s)
I20260809 23:04:34.291200 17988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:34.292619 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 17988
I20260809 23:04:34.292809 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:34.293452 24703 external_mini_cluster.cc:1428] Setting key f61391509fefd6c5fa4b4310d5e489bd
I20260809 23:04:34.295995 18061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:34.301384 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:34.295343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45251 (local address 127.24.31.254:32951)
0809 23:04:34.296186 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:34.296197 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:34.296235 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:04:34.297944 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:34.297965 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:34.298303 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:34.298635 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:34.299750 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:34.299757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:34.299768 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:34.299783 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:34.299872 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:34.299891 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:34.300064 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:34.300173 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 23:04:34.300399 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":400,"threads_started":1}
I20260809 23:04:34.304926 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:34.308908 18061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b: Bootstrap starting.
I20260809 23:04:34.313098 18061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:34.313362 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:34.310622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55879 (local address 127.24.31.254:32951)
0809 23:04:34.310778 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:34.310782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:34.311221 (+ 439us) server_negotiation.cc:407] Connection header received
0809 23:04:34.311386 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:34.311390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:34.311439 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:34.311533 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:34.312479 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:34.312483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:34.312486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:04:34.312501 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:34.312540 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:04:34.312556 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:34.312654 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:34.312808 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 23:04:34.312856 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 23:04:34.314369 18061 log.cc:824] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:34.317070 18061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b: No bootstrap required, opened a new log
I20260809 23:04:34.324324 18061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8298df64d644bd9e3f4cc770bd955b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 32951 } }
I20260809 23:04:34.324668 18061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:04:34.324817 18061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b8298df64d644bd9e3f4cc770bd955b, State: Initialized, Role: FOLLOWER
I20260809 23:04:34.325467 18061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8298df64d644bd9e3f4cc770bd955b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 32951 } }
I20260809 23:04:34.325706 18061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:04:34.325873 18061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:04:34.326053 18061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:34.328240 18061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8298df64d644bd9e3f4cc770bd955b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 32951 } }
I20260809 23:04:34.328781 18061 leader_election.cc:302] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b8298df64d644bd9e3f4cc770bd955b; no voters:
I20260809 23:04:34.329576 18061 leader_election.cc:288] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:04:34.329682 18074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:34.330602 18074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [term 1 LEADER]: Becoming Leader. State: Replica: 5b8298df64d644bd9e3f4cc770bd955b, State: Running, Role: LEADER
I20260809 23:04:34.331198 18074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8298df64d644bd9e3f4cc770bd955b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 32951 } }
I20260809 23:04:34.331905 18061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:34.334241 18075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b8298df64d644bd9e3f4cc770bd955b. Latest consensus state: current_term: 1 leader_uuid: "5b8298df64d644bd9e3f4cc770bd955b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8298df64d644bd9e3f4cc770bd955b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 32951 } } }
I20260809 23:04:34.334218 18076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b8298df64d644bd9e3f4cc770bd955b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b8298df64d644bd9e3f4cc770bd955b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 32951 } } }
I20260809 23:04:34.334789 18075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:34.334827 18076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:34.342398 18079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:04:34.347620 18079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:34.356443 18079 catalog_manager.cc:1347] Generated new cluster ID: 173f02833cce4632a25f99ecab942f70
I20260809 23:04:34.356567 18079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:34.379422 18079 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:34.379889 18079 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:34.389326 18079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b8298df64d644bd9e3f4cc770bd955b: Generated new TSK 0
I20260809 23:04:34.390051 18079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:34.400485 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17988
W20260809 23:04:34.401571 18056 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45251
W20260809 23:04:34.402293 24703 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 17988: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 23:04:34.402452 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 17988
2026-08-09T23:04:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2048 ms)
[----------] 18 tests from AuthTokenIssuingTest (37269 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:34 dist-test-slave-vxj2 krb5kdc[18099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:34 dist-test-slave-vxj2 krb5kdc[18099](info): set up 2 sockets
Aug 09 23:04:34 dist-test-slave-vxj2 krb5kdc[18099](info): commencing operation
krb5kdc: starting...
W20260809 23:04:35.599411 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.003s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:35 dist-test-slave-vxj2 krb5kdc[18099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:04:35.901666 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39897
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39897,127.24.31.253:45045,127.24.31.252:39111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:36.235304 18112 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:04:36.235669 18112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:36.235854 18112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:36.236011 18112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:36.243791 18112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:36.243889 18112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:36.243978 18112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:36.244059 18112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:36.244170 18112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:36.254474 18112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39897,127.24.31.253:45045,127.24.31.252:39111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:39897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:36.255903 18112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:36.257287 18112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:36.269598 18120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:36.269558 18126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:36.269634 18122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:36.270431 18112 server_base.cc:1034] running on GCE node
I20260809 23:04:36.271202 18112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:36.272596 18112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:36.273849 18112 hybrid_clock.cc:648] HybridClock initialized: now 1786316676273793 us; error 41 us; skew 500 ppm
I20260809 23:04:36.274187 18112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:36 dist-test-slave-vxj2 krb5kdc[18099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316676, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:36.277515 18112 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:04:36.278991 18112 webserver.cc:460] Webserver started at http://127.24.31.254:34157/ using document root <none> and password file <none>
I20260809 23:04:36.279702 18112 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:36.279847 18112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:36.280200 18112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:36.284009 18112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "e3ff8c1c2aa34d73898a49ff8275056b"
format_stamp: "Formatted at 2026-08-09 23:04:36 on dist-test-slave-vxj2"
server_key: "779a4708dd3e30f4aeeda3239367811c"
server_key_iv: "7a4692fd0cbc1db987ceccb37e66f115"
server_key_version: "encryptionkey@0"
I20260809 23:04:36.284847 18112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "e3ff8c1c2aa34d73898a49ff8275056b"
format_stamp: "Formatted at 2026-08-09 23:04:36 on dist-test-slave-vxj2"
server_key: "779a4708dd3e30f4aeeda3239367811c"
server_key_iv: "7a4692fd0cbc1db987ceccb37e66f115"
server_key_version: "encryptionkey@0"
I20260809 23:04:36.290238 18112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 23:04:36.293823 18134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:36.295207 18112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 23:04:36.295404 18112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "e3ff8c1c2aa34d73898a49ff8275056b"
format_stamp: "Formatted at 2026-08-09 23:04:36 on dist-test-slave-vxj2"
server_key: "779a4708dd3e30f4aeeda3239367811c"
server_key_iv: "7a4692fd0cbc1db987ceccb37e66f115"
server_key_version: "encryptionkey@0"
I20260809 23:04:36.295632 18112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:36.307235 18112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:36.311487 18112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:36.311775 18112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:36.330983 18112 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:39897
I20260809 23:04:36.331002 18188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:39897 every 8 connection(s)
I20260809 23:04:36.332485 18112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:36.336308 18189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:36.341274 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 18112
I20260809 23:04:36.341507 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:36.342357 24703 external_mini_cluster.cc:1428] Setting key 5db06d22f7141ade84c78909b94dab36
I20260809 23:04:36.344061 18189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:04:36 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316676, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:04:36.371438 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.348103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39897 (local address 127.0.0.1:52532)
0809 23:04:36.349029 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.349077 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:36.351925 (+ 2848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:36.352779 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:36.352788 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:36.353659 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:36.354165 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:36.354191 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.356494 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.356498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:36.357674 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:36.357686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.359107 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.359112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:36.359235 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:36.359915 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:36.359937 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:36.362296 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:36.367130 (+ 4834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:36.367141 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:36.367151 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:36.367580 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:36.368104 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:36.368107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:36.368109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:36.368269 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:36.369363 (+ 1094us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:36.369372 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:36.369583 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 23:04:36.369813 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 23:04:36.370225 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":370,"threads_started":1}
I20260809 23:04:36.371488 18192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.348103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.24.31.254:39897)
0809 23:04:36.350046 (+ 1943us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:36.350054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:36.350095 (+ 41us) server_negotiation.cc:407] Connection header received
0809 23:04:36.352283 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:36.352289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:36.352512 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:36.352677 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:36.354392 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.356328 (+ 1936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.357848 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.358874 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.359061 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:36.362562 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:36.362574 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:36.362593 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:36.362635 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:36.366986 (+ 4351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:36.367747 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:36.367759 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:36.367765 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:36.367981 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:36.368395 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:36.368403 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:36.368409 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:36.369221 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:36.369474 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:36.370020 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 23:04:36.370448 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":1759,"thread_start_us":1515,"threads_started":1}
I20260809 23:04:36.374622 18189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.31.253@KRBTEST.COM; defaulting to no policy
I20260809 23:04:36.375669 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.375239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:35968)
0809 23:04:36.375437 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.375540 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 23:04:36.375953 18136 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
I20260809 23:04:36.377825 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.377346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:35976)
0809 23:04:36.377537 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.377648 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 23:04:36.378046 18189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:45045: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab.
Entry for principal kudu/127.24.31.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab.
I20260809 23:04:36.412866 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.253:45045
--webserver_interface=127.24.31.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab
--principal=kudu/127.24.31.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39897,127.24.31.253:45045,127.24.31.252:39111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:04:36.427448 18189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } attempt: 1
I20260809 23:04:36.428870 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.428241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:35988)
0809 23:04:36.428524 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.428677 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 23:04:36.430039 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.429571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:35992)
0809 23:04:36.429747 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.429846 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 23:04:36.430225 18189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:45045: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
I20260809 23:04:36.498535 18189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } attempt: 2
I20260809 23:04:36.500023 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.499365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:36000)
0809 23:04:36.499679 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.499823 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 23:04:36.501358 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.500822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:36014)
0809 23:04:36.500997 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.501118 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 23:04:36.501569 18189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:45045: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
I20260809 23:04:36.592886 18189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } attempt: 3
I20260809 23:04:36.594359 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.593787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:36030)
0809 23:04:36.594050 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.594175 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 23:04:36.595566 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.595106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:36032)
0809 23:04:36.595279 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.595376 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 23:04:36.595810 18189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:45045: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
I20260809 23:04:36.739132 18189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } attempt: 4
I20260809 23:04:36.740520 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.739869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:36044)
0809 23:04:36.740201 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.740338 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 23:04:36.741806 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.741281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:36054)
0809 23:04:36.741458 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.741586 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 23:04:36.742051 18189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:45045: Network error: Client connection negotiation failed: client connection to 127.24.31.253:45045: connect: Connection refused (error 111)
W20260809 23:04:36.758176 18197 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:04:36.758579 18197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:36.758713 18197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:36.758864 18197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:36.766520 18197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:36.766623 18197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:36.766768 18197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:36.766858 18197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:36.766988 18197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:36.777132 18197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39897,127.24.31.253:45045,127.24.31.252:39111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab
--principal=kudu/127.24.31.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.253:45045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.31.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:36.778599 18197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:36.779976 18197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:36.791978 18203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:36.792084 18204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:36.792279 18206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:36.793494 18197 server_base.cc:1034] running on GCE node
I20260809 23:04:36.794332 18197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:36.795642 18197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:36.796905 18197 hybrid_clock.cc:648] HybridClock initialized: now 1786316676796824 us; error 51 us; skew 500 ppm
I20260809 23:04:36.797343 18197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:36 dist-test-slave-vxj2 krb5kdc[18099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316676, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:36.800997 18197 init.cc:377] Logged in from keytab as kudu/127.24.31.253@KRBTEST.COM (short username kudu)
I20260809 23:04:36.802536 18197 webserver.cc:460] Webserver started at http://127.24.31.253:40781/ using document root <none> and password file <none>
I20260809 23:04:36.803267 18197 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:36.803395 18197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:36.803771 18197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:36.806998 18197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/data/instance:
uuid: "34c61e48fb86411a8429e9c6b7980a55"
format_stamp: "Formatted at 2026-08-09 23:04:36 on dist-test-slave-vxj2"
server_key: "2b34977e0a1f1d6c6b889ccd6b1eca01"
server_key_iv: "4d1627f979e880825e7c498826126d6c"
server_key_version: "encryptionkey@0"
I20260809 23:04:36.807875 18197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/wal/instance:
uuid: "34c61e48fb86411a8429e9c6b7980a55"
format_stamp: "Formatted at 2026-08-09 23:04:36 on dist-test-slave-vxj2"
server_key: "2b34977e0a1f1d6c6b889ccd6b1eca01"
server_key_iv: "4d1627f979e880825e7c498826126d6c"
server_key_version: "encryptionkey@0"
I20260809 23:04:36.813246 18197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 23:04:36.816717 18214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:36.817977 18197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:04:36.818159 18197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/wal
uuid: "34c61e48fb86411a8429e9c6b7980a55"
format_stamp: "Formatted at 2026-08-09 23:04:36 on dist-test-slave-vxj2"
server_key: "2b34977e0a1f1d6c6b889ccd6b1eca01"
server_key_iv: "4d1627f979e880825e7c498826126d6c"
server_key_version: "encryptionkey@0"
I20260809 23:04:36.818398 18197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:36.838837 18197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:36.842794 18197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:36.843092 18197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:36.861990 18197 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.253:45045
I20260809 23:04:36.862010 18267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.253:45045 every 8 connection(s)
I20260809 23:04:36.863305 18197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
I20260809 23:04:36.867081 18268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:36.871641 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 18197
I20260809 23:04:36.871830 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-1/wal/instance
I20260809 23:04:36.872380 24703 external_mini_cluster.cc:1428] Setting key 011ebd542035374641a2b6e74134e02b
I20260809 23:04:36.874672 18268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:04:36 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316676, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:04:36.899077 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.878016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52548 (local address 127.24.31.254:39897)
0809 23:04:36.878579 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:36.878586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:36.879569 (+ 983us) server_negotiation.cc:407] Connection header received
0809 23:04:36.882423 (+ 2854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:36.882427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:36.882524 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:36.882629 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:36.884654 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.885549 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.887394 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.888273 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.888438 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:36.891908 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:36.891917 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:36.891925 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:36.891977 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:36.895756 (+ 3779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:36.896489 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:36.896497 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:36.896503 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:36.896649 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:36.897028 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:36.897036 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:36.897042 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:36.897475 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:36.897641 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:36.898284 (+ 643us) server_negotiation.cc:299] Negotiation successful
0809 23:04:36.898531 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":280,"threads_started":1}
I20260809 23:04:36.899945 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.878516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39897 (local address 127.0.0.1:52548)
0809 23:04:36.879373 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.879434 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:36.882114 (+ 2680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:36.882896 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:36.882910 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:36.883671 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:36.884430 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:36.884458 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.885720 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.885728 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:36.887230 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:36.887243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.888524 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.888529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:36.888706 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:36.889429 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:36.889454 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:36.891625 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:36.895924 (+ 4299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:36.895930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:36.895941 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:36.896362 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:36.896759 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:36.896762 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:36.896764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:36.896921 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:36.897601 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:36.897606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:36.897921 (+ 315us) client_negotiation.cc:769] Sending connection context
0809 23:04:36.898155 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 23:04:36.898580 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":331,"threads_started":1}
I20260809 23:04:36.901868 18268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.31.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.252@KRBTEST.COM" created.
Aug 09 23:04:36 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316676, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
I20260809 23:04:36.917572 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.902601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:36062)
0809 23:04:36.902791 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.902810 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:36.902890 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:36.903919 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:36.903923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:36.904341 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:36.904460 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:36.904467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.905865 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.905872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:36.906706 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:36.906717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.907493 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.907498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:36.907715 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:36.908268 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:36.908294 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:36.910646 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:36.915141 (+ 4495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:36.915145 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:36.915147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:36.915535 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:36.915946 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:36.915949 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:36.915951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:36.916077 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:36.916820 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:36.916823 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:36.916939 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 23:04:36.917046 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 23:04:36.917319 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":3}
I20260809 23:04:36.918079 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.902908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.24.31.253:45045)
0809 23:04:36.903513 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:36.903518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:36.903541 (+ 23us) server_negotiation.cc:407] Connection header received
0809 23:04:36.903578 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:36.903583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:36.903754 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:36.903929 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:36.904629 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.905733 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.906848 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:36.907355 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:36.907486 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:36.910883 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:36.910899 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:36.910917 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:36.910963 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:36.914982 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:36.915675 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:36.915685 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:36.915692 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:36.915835 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:36.916180 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:36.916187 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:36.916193 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:36.916668 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:36.916883 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:36.917252 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 23:04:36.917539 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":463,"thread_start_us":336,"threads_started":1}
I20260809 23:04:36.920400 18268 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:04:36.921563 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.921063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54280)
0809 23:04:36.921310 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.921420 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260809 23:04:36.921912 18215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
I20260809 23:04:36.923599 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.923184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54286)
0809 23:04:36.923376 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.923449 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 23:04:36.923919 18268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:39111: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab.
Entry for principal kudu/127.24.31.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab.
I20260809 23:04:36.945119 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.252:39111
--webserver_interface=127.24.31.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab
--principal=kudu/127.24.31.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:39897,127.24.31.253:45045,127.24.31.252:39111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:04:36.973287 18268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } attempt: 1
I20260809 23:04:36.974520 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.974063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54296)
0809 23:04:36.974253 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.974370 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 23:04:36.975566 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:36.975183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54312)
0809 23:04:36.975349 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:36.975445 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 23:04:36.975805 18268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:39111: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
I20260809 23:04:37.041383 18189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } attempt: 5
I20260809 23:04:37.044065 18268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } attempt: 2
I20260809 23:04:37.045143 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.044627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54324)
0809 23:04:37.044853 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.044957 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 23:04:37.046396 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.045970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54338)
0809 23:04:37.046140 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.046230 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 23:04:37.046804 18268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:39111: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316676, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
I20260809 23:04:37.056108 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.042197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:36072)
0809 23:04:37.042440 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.042462 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:37.042635 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:37.043078 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:37.043083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:37.043505 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:37.043688 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.043696 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.044956 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.044962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.046166 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.046183 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.047384 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.047389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.047555 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.048180 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:37.048197 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:37.050210 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:37.053933 (+ 3723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.053936 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.053939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.054306 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.054695 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.054698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.054700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.054839 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.055420 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:37.055423 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:37.055553 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 23:04:37.055661 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 23:04:37.055878 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 23:04:37.056305 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.042299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36072 (local address 127.24.31.253:45045)
0809 23:04:37.042530 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.042538 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.042563 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:04:37.042853 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.042858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.042924 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.043039 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.043854 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.044826 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.046345 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.047230 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.047388 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.050411 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.050418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.050423 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.050461 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.053789 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.054427 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.054432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.054436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.054580 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.054948 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.054953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.054956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.055306 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.055454 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.055810 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.056027 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 23:04:37.057441 18189 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:04:37.058537 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.058155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54344)
0809 23:04:37.058337 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.058409 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 23:04:37.059449 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.059129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54358)
0809 23:04:37.059288 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.059348 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 23:04:37.059665 18189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:39111: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
I20260809 23:04:37.110888 18189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } attempt: 1
I20260809 23:04:37.112035 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.111542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54360)
0809 23:04:37.111770 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.111872 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 23:04:37.113113 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.112797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54366)
0809 23:04:37.112953 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.113019 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 23:04:37.113420 18189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:39111: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
I20260809 23:04:37.138067 18268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } attempt: 3
I20260809 23:04:37.139372 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.138726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54376)
0809 23:04:37.139002 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.139142 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 23:04:37.140856 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.140280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54392)
0809 23:04:37.140597 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.140696 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210}
W20260809 23:04:37.141245 18268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:39111: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
I20260809 23:04:37.181663 18189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } attempt: 2
I20260809 23:04:37.183014 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.182468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54396)
0809 23:04:37.182705 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.182837 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 23:04:37.184109 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.183781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54402)
0809 23:04:37.183954 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.184017 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 23:04:37.184373 18189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:39111: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
W20260809 23:04:37.281111 18277 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:04:37.281520 18277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:37.281641 18277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:37.281773 18277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 23:04:37.284508 18268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } attempt: 4
I20260809 23:04:37.285826 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.285287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54412)
0809 23:04:37.285535 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.285647 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 23:04:37.286965 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.286570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54422)
0809 23:04:37.286779 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.286857 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 23:04:37.287230 18268 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:39111: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
W20260809 23:04:37.289620 18277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:37.289726 18277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:37.289844 18277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:37.289947 18277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:37.290036 18277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:37.290558 18189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } attempt: 3
I20260809 23:04:37.291560 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.291111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54428)
0809 23:04:37.291303 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.291397 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 23:04:37.292779 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.292313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54434)
0809 23:04:37.292534 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.292627 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260809 23:04:37.293092 18189 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:39111: Network error: Client connection negotiation failed: client connection to 127.24.31.252:39111: connect: Connection refused (error 111)
I20260809 23:04:37.300652 18277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:39897,127.24.31.253:45045,127.24.31.252:39111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab
--principal=kudu/127.24.31.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.252:39111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.31.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:37.302004 18277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:37.303298 18277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:37.315415 18282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:37.315469 18287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:37.315451 18284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:37.318801 18277 server_base.cc:1034] running on GCE node
I20260809 23:04:37.319669 18277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:37.321250 18277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:37.322576 18277 hybrid_clock.cc:648] HybridClock initialized: now 1786316677322478 us; error 40 us; skew 500 ppm
I20260809 23:04:37.322948 18277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316677, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:37.329911 18277 init.cc:377] Logged in from keytab as kudu/127.24.31.252@KRBTEST.COM (short username kudu)
I20260809 23:04:37.331699 18277 webserver.cc:460] Webserver started at http://127.24.31.252:44037/ using document root <none> and password file <none>
I20260809 23:04:37.332446 18277 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:37.332576 18277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:37.332885 18277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:37.335886 18277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/data/instance:
uuid: "529ad8978ff54c0d98ca4eee4be863c3"
format_stamp: "Formatted at 2026-08-09 23:04:37 on dist-test-slave-vxj2"
server_key: "5c198bb615d2f3281785a9f3d7ee30c7"
server_key_iv: "7d4aec70c3ac19818fe3fd4a6753aa16"
server_key_version: "encryptionkey@0"
I20260809 23:04:37.336678 18277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/wal/instance:
uuid: "529ad8978ff54c0d98ca4eee4be863c3"
format_stamp: "Formatted at 2026-08-09 23:04:37 on dist-test-slave-vxj2"
server_key: "5c198bb615d2f3281785a9f3d7ee30c7"
server_key_iv: "7d4aec70c3ac19818fe3fd4a6753aa16"
server_key_version: "encryptionkey@0"
I20260809 23:04:37.341787 18277 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 23:04:37.345345 18294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:37.346616 18277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:04:37.346801 18277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/wal
uuid: "529ad8978ff54c0d98ca4eee4be863c3"
format_stamp: "Formatted at 2026-08-09 23:04:37 on dist-test-slave-vxj2"
server_key: "5c198bb615d2f3281785a9f3d7ee30c7"
server_key_iv: "7d4aec70c3ac19818fe3fd4a6753aa16"
server_key_version: "encryptionkey@0"
I20260809 23:04:37.347013 18277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:37.358759 18277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:37.362640 18277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:37.362907 18277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:37.382431 18277 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.252:39111
I20260809 23:04:37.382452 18355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.252:39111 every 8 connection(s)
I20260809 23:04:37.383735 18277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
I20260809 23:04:37.388255 18356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:37.392504 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 18277
I20260809 23:04:37.392705 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316311377261-24703-0/minicluster-data/master-2/wal/instance
I20260809 23:04:37.393271 24703 external_mini_cluster.cc:1428] Setting key 7633a19c3ff8d9023daf83d9fdc41aed
I20260809 23:04:37.396471 18356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316677, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:04:37.415198 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.400710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.24.31.254:39897)
0809 23:04:37.401223 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.401231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.401694 (+ 463us) server_negotiation.cc:407] Connection header received
0809 23:04:37.401910 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.401915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.402007 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.402121 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.403246 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.403958 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.405561 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.406206 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.406374 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.409680 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.409685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.409688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.409728 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.412308 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.413001 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.413006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.413010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.413137 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.413579 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.413584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.413588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.413866 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.414029 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.414582 (+ 553us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.414795 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":271,"threads_started":1}
I20260809 23:04:37.416195 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.395100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54446 (local address 127.24.31.252:39111)
0809 23:04:37.395923 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.395937 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.395984 (+ 47us) server_negotiation.cc:407] Connection header received
0809 23:04:37.397852 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.397882 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.398387 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.398813 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.400639 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.402490 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.403846 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.404504 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.404716 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.409288 (+ 4572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.409296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.409306 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.409345 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.412299 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.413017 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.413031 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.413033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.413252 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.413640 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.413645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.413648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.414125 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.414459 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.414783 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.415120 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":589,"thread_start_us":349,"threads_started":1}
I20260809 23:04:37.416213 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.400660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:39897 (local address 127.0.0.1:52564)
0809 23:04:37.401560 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.401599 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:37.401780 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:37.402180 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:37.402186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:37.402851 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:37.403085 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.403094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.404120 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.404132 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.405370 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.405387 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.406437 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.406441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.406572 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.407175 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:37.407195 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:37.409469 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:37.412447 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.412458 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.412468 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.412872 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.413286 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.413291 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.413295 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.413458 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.413996 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:37.414003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:37.414213 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 23:04:37.414465 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 23:04:37.414965 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":561,"threads_started":1}
I20260809 23:04:37.417865 18356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316677, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:04:37.431420 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.418484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:45045 (local address 127.0.0.1:36084)
0809 23:04:37.418727 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.418757 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:37.418869 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:37.419213 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:37.419219 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:37.419684 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:37.419839 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.419847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.420817 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.420825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.421803 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.421818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.422700 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.422704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.422828 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.423435 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:37.423450 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:37.425557 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:37.428854 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.428860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.428863 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.429406 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.429813 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.429818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.429821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.429974 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.430505 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:37.430509 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:37.430662 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 23:04:37.430800 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 23:04:37.431059 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260809 23:04:37.431453 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.418628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36084 (local address 127.24.31.253:45045)
0809 23:04:37.418798 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.418803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.418851 (+ 48us) server_negotiation.cc:407] Connection header received
0809 23:04:37.418975 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.418979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.419057 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.419167 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.419994 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.420681 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.421948 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.422377 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.422513 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.425761 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.425766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.425769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.425800 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.428697 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.429537 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.429543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.429546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.429681 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.430090 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.430094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.430099 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.430350 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.430520 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.430901 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.431094 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 23:04:37.432631 18356 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:04:37.432888 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.419559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52566 (local address 127.24.31.254:39897)
0809 23:04:37.419721 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.419728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.419747 (+ 19us) server_negotiation.cc:407] Connection header received
0809 23:04:37.419946 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.419951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.420026 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.420124 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.421218 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.421976 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.423202 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.423597 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.423723 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.427767 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.427771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.427774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.427801 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.430394 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.431000 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.431007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.431010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.431127 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.431490 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.431498 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.431504 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.431813 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.432099 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.432427 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.432613 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316677, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
I20260809 23:04:37.446363 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.433442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54448)
0809 23:04:37.433593 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.433611 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:37.433700 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:37.434142 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:37.434145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:37.434606 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:37.434793 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.434807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.435850 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.435857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.436963 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.436975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.437803 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.437810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.437999 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.438543 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:37.438566 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:37.440866 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:37.443885 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.443888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.443891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.444228 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.444628 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.444631 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.444634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.444786 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.445605 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:37.445608 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:37.445726 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 23:04:37.445873 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 23:04:37.446092 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 23:04:37.446666 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.433677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.24.31.252:39111)
0809 23:04:37.433859 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.433865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.433883 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:04:37.433952 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.433955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.434017 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.434110 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.434905 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.435716 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.437098 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.437621 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.437709 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.441044 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.441049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.441052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.441077 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.443720 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.444348 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.444356 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.444363 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.444517 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.444899 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.444910 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.444913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.445305 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.445455 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.446011 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.446268 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 23:04:37.449839 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.434548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36088 (local address 127.24.31.253:45045)
0809 23:04:37.434688 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.434692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.434816 (+ 124us) server_negotiation.cc:407] Connection header received
0809 23:04:37.434989 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.434994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.435063 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.435162 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.436454 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.437260 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.438792 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.439220 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.439312 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.443887 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.443894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.443898 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.443939 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.447001 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.447846 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.447854 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.447860 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.448000 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.448434 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.448440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.448443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.448710 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.449007 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.449306 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.449529 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:04:37.452257 18356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3: Bootstrap starting.
I20260809 23:04:37.455262 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:37.456033 18356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:37.457041 18356 log.cc:824] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:37.460203 18356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3: No bootstrap required, opened a new log
I20260809 23:04:37.458163 18163 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e3ff8c1c2aa34d73898a49ff8275056b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52566
I20260809 23:04:37.470316 18189 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } attempt: 4
I20260809 23:04:37.470991 18356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } }
I20260809 23:04:37.471441 18356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529ad8978ff54c0d98ca4eee4be863c3, State: Initialized, Role: FOLLOWER
I20260809 23:04:37.471961 18356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } }
I20260809 23:04:37.473028 18365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.473481 18365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:37.474203 18356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316676, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
W20260809 23:04:37.485428 18377 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:04:37.485575 18377 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:04:37.488515 18191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.471030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54462)
0809 23:04:37.471230 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.471262 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:37.471381 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:37.471776 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:37.471781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:37.472281 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:37.472543 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.472561 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.474007 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.474013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.475523 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.475542 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.476929 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.476935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.477202 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.477868 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:37.477906 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:37.480885 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:37.485518 (+ 4633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.485524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.485528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.486048 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.486551 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.486557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.486560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.486738 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.487513 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:37.487518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:37.487689 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 23:04:37.487817 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 23:04:37.488100 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 23:04:37.489661 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.471231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54462 (local address 127.24.31.252:39111)
0809 23:04:37.471454 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.471460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.471480 (+ 20us) server_negotiation.cc:407] Connection header received
0809 23:04:37.471532 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.471537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.471615 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.471747 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.472750 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.473862 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.475733 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.476725 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.476874 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.481070 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.481076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.481083 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.481120 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.485344 (+ 4224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.486216 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.486222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.486226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.486392 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.486888 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.486893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.486896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.487354 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.488902 (+ 1548us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.489024 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.489289 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 23:04:37.481953 18277 master_runner.cc:186] Error getting master registration for 127.24.31.254:39897: OK, instance_id { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" instance_seqno: 1786316676325380 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:04:37.493834 18277 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:04:37.495093 18189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b: Bootstrap starting.
I20260809 23:04:37.498049 18189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:37.499078 18189 log.cc:824] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:37.501947 18189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b: No bootstrap required, opened a new log
I20260809 23:04:37.506930 18189 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } }
I20260809 23:04:37.507216 18189 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3ff8c1c2aa34d73898a49ff8275056b, State: Initialized, Role: FOLLOWER
I20260809 23:04:37.507617 18189 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } }
I20260809 23:04:37.508507 18380 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.508898 18380 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:37.509840 18189 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:37.509943 18163 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 23:04:37.510128 18163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:37.514295 18163 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } }
I20260809 23:04:37.516152 18163 leader_election.cc:288] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [CANDIDATE]: Term 1 election: Requested vote from peers 34c61e48fb86411a8429e9c6b7980a55 (127.24.31.253:45045), 529ad8978ff54c0d98ca4eee4be863c3 (127.24.31.252:39111)
I20260809 23:04:37.517475 18329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "529ad8978ff54c0d98ca4eee4be863c3"
I20260809 23:04:37.518080 18329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:37.521800 18329 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3ff8c1c2aa34d73898a49ff8275056b in term 1.
I20260809 23:04:37.522385 18135 leader_election.cc:302] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 529ad8978ff54c0d98ca4eee4be863c3, e3ff8c1c2aa34d73898a49ff8275056b; no voters:
I20260809 23:04:37.522841 18380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:37.533460 18380 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [term 1 LEADER]: Becoming Leader. State: Replica: e3ff8c1c2aa34d73898a49ff8275056b, State: Running, Role: LEADER
I20260809 23:04:37.534991 18380 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } }
I20260809 23:04:37.549857 18387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ff8c1c2aa34d73898a49ff8275056b. Latest consensus state: current_term: 1 leader_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.551051 18387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:37.561262 18396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260809 23:04:37.564661 18140 tablet.cc:2369] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:04:37.583279 18396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:37.586412 18268 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } attempt: 5
I20260809 23:04:37.587309 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.553089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.24.31.254:39897)
0809 23:04:37.553319 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.553329 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.564015 (+ 10686us) server_negotiation.cc:407] Connection header received
0809 23:04:37.565813 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.565824 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.565945 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.566049 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.568863 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.570050 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.572311 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.573117 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.573276 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.578779 (+ 5503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.578787 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.578793 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.578839 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.583011 (+ 4172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.584012 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.584019 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.584023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.584205 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.584706 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.584726 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.584731 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.585208 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.585513 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.586158 (+ 645us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.586457 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 23:04:37.605760 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.564729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.24.31.253:45045)
0809 23:04:37.564942 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.564948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.578474 (+ 13526us) server_negotiation.cc:407] Connection header received
0809 23:04:37.579051 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.579060 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.579154 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.579245 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.583539 (+ 4294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.584696 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.587661 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.588388 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.588505 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.595675 (+ 7170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.595684 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.595689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.595745 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.599565 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.600460 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.600466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.600470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.600675 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.601254 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.601259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.601263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.601670 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.601944 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.604445 (+ 2501us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.604922 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 23:04:37.613137 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.559923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54470 (local address 127.24.31.252:39111)
0809 23:04:37.560185 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.560195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.568941 (+ 8746us) server_negotiation.cc:407] Connection header received
0809 23:04:37.569982 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.569992 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.570095 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.570177 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.574808 (+ 4631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.575867 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.579487 (+ 3620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.580150 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.580271 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.603155 (+ 22884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.603162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.603167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.603214 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.606889 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.608955 (+ 2066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.608960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.608964 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.609136 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.610373 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.610379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.610382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.610731 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.611068 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.612459 (+ 1391us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.612732 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260809 23:04:37.615049 18301 tablet.cc:2369] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 23:04:37 dist-test-slave-vxj2 krb5kdc[18099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316676, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
I20260809 23:04:37.624692 18329 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [term 1 FOLLOWER]: Refusing update from remote peer e3ff8c1c2aa34d73898a49ff8275056b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:04:37.625779 18380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [LEADER]: Connected to new peer: Peer: permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:04:37.636186 18272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.587001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:39111 (local address 127.0.0.1:54484)
0809 23:04:37.587677 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:37.587698 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:37.587808 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:37.613812 (+ 26004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:37.613817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:37.614334 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:37.614552 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.614568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.616172 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.616177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.617624 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:37.617638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.618690 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.618696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:37.618965 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.619675 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:37.619704 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:37.622974 (+ 3270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:37.631824 (+ 8850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.631830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.631835 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.632404 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.633392 (+ 988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:37.633397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:37.633400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:37.633593 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:37.635045 (+ 1452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:37.635052 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:37.635220 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 23:04:37.635383 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 23:04:37.635690 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551}
I20260809 23:04:37.639267 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.588026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.24.31.252:39111)
0809 23:04:37.613484 (+ 25458us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.613489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.613508 (+ 19us) server_negotiation.cc:407] Connection header received
0809 23:04:37.613555 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.613559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.613630 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.614712 (+ 1082us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.614851 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.615988 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.617816 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.618511 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.619311 (+ 800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.623653 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:37.623739 (+ 86us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:37.623748 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:37.623826 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:37.631558 (+ 7732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.632687 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.632692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.632696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.633126 (+ 430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:37.633879 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:37.633884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:37.633889 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:37.634806 (+ 917us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:37.636850 (+ 2044us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.637135 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.638251 (+ 1116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25356,"thread_start_us":35414,"threads_started":1}
I20260809 23:04:37.642524 18404 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 23:04:37.645926 18268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55: Bootstrap starting.
I20260809 23:04:37.647003 18365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ff8c1c2aa34d73898a49ff8275056b. Latest consensus state: current_term: 1 leader_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.647317 18365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:37.654052 18268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:37.656519 18268 log.cc:824] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:37.648895 18112 master_runner.cc:186] Error getting master registration for 127.24.31.253:45045: OK, instance_id { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" instance_seqno: 1786316676856841 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:04:37.667411 18365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.667657 18365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:37.672650 18268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55: No bootstrap required, opened a new log
I20260809 23:04:37.670576 18387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.674001 18387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:37.674073 18112 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:04:37.674994 18396 catalog_manager.cc:1347] Generated new cluster ID: f8caf0480e1f4f6791b92b29df53bc17
I20260809 23:04:37.675092 18396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:37.684649 18268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } }
I20260809 23:04:37.685101 18268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c61e48fb86411a8429e9c6b7980a55, State: Initialized, Role: FOLLOWER
I20260809 23:04:37.685657 18268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } }
I20260809 23:04:37.692725 18413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.694886 18268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:37.694975 18243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "34c61e48fb86411a8429e9c6b7980a55"
I20260809 23:04:37.695442 18243 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:37.696309 18413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:37.699542 18243 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3ff8c1c2aa34d73898a49ff8275056b in term 1.
I20260809 23:04:37.700307 18242 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [term 1 FOLLOWER]: Refusing update from remote peer e3ff8c1c2aa34d73898a49ff8275056b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:04:37.701308 18387 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [LEADER]: Connected to new peer: Peer: permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:04:37.716825 18396 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:37.718827 18396 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:37.742561 18387 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.742859 18387 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:37.742492 18413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3ff8c1c2aa34d73898a49ff8275056b. Latest consensus state: current_term: 1 leader_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.742991 18413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:37.745226 18420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3ff8c1c2aa34d73898a49ff8275056b" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 39897 } } peers { permanent_uuid: "34c61e48fb86411a8429e9c6b7980a55" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 45045 } } peers { permanent_uuid: "529ad8978ff54c0d98ca4eee4be863c3" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 39111 } } }
I20260809 23:04:37.745667 18420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:37.758227 18396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3ff8c1c2aa34d73898a49ff8275056b: Generated new TSK 0
I20260809 23:04:37.759362 18396 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:37.782480 18429 catalog_manager.cc:1259] Loaded cluster ID: f8caf0480e1f4f6791b92b29df53bc17
I20260809 23:04:37.782660 18429 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55: loading cluster ID for follower catalog manager: success
I20260809 23:04:37.786553 18429 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55: acquiring CA information for follower catalog manager: success
I20260809 23:04:37.789450 18429 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 34c61e48fb86411a8429e9c6b7980a55: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 23:04:37.924819 18361 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:37.910672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52588 (local address 127.24.31.254:39897)
0809 23:04:37.910989 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.910999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.921579 (+ 10580us) server_negotiation.cc:407] Connection header received
0809 23:04:37.923123 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.923130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.923223 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.923297 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.924573 (+ 1276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52588: BlockingRecv error: recv got EOF from 127.0.0.1:52588 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 23:04:37.928023 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:37.917502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54488 (local address 127.24.31.252:39111)
0809 23:04:37.917740 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.917751 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.926297 (+ 8546us) server_negotiation.cc:407] Connection header received
0809 23:04:37.926454 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.926483 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.926593 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.926709 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.927718 (+ 1009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54488: BlockingRecv error: recv got EOF from 127.0.0.1:54488 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 23:04:37.930440 18274 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:37.922303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36114 (local address 127.24.31.253:45045)
0809 23:04:37.922482 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.922487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.927798 (+ 5311us) server_negotiation.cc:407] Connection header received
0809 23:04:37.929071 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.929077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.929213 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.929290 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:37.930215 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36114: BlockingRecv error: recv got EOF from 127.0.0.1:36114 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 23:04:37.988685 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:37.969538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36120 (local address 127.24.31.253:45045)
0809 23:04:37.969747 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:37.969757 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:37.980346 (+ 10589us) server_negotiation.cc:407] Connection header received
0809 23:04:37.980931 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:37.980939 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:37.981146 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:37.981329 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:37.982075 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.983264 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.985436 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:37.986136 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:37.986235 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:37.987068 (+ 833us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:37.987767 (+ 699us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:37.987893 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:37.988201 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 23:04:37.988312 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 23:04:38.036091 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:38.022251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54498 (local address 127.24.31.252:39111)
0809 23:04:38.022470 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.022480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.033828 (+ 11348us) server_negotiation.cc:407] Connection header received
0809 23:04:38.034272 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.034281 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.034421 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.034522 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:38.035776 (+ 1254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54498: BlockingRecv error: recv got EOF from 127.0.0.1:54498 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:04:38.087672 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.069548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36128 (local address 127.24.31.253:45045)
0809 23:04:38.069740 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.069746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.080379 (+ 10633us) server_negotiation.cc:407] Connection header received
0809 23:04:38.080979 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.080986 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.081090 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.081192 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.081922 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.083117 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.085232 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.085963 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.086066 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.086744 (+ 678us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.086902 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.086987 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.087233 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.087324 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 23:04:38.134843 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:38.121787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54508 (local address 127.24.31.252:39111)
0809 23:04:38.121988 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.121995 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.132610 (+ 10615us) server_negotiation.cc:407] Connection header received
0809 23:04:38.133208 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.133215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.133316 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.133409 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:38.134574 (+ 1165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54508: BlockingRecv error: recv got EOF from 127.0.0.1:54508 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 23:04:38.189795 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.171974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36142 (local address 127.24.31.253:45045)
0809 23:04:38.172169 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.172177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.182771 (+ 10594us) server_negotiation.cc:407] Connection header received
0809 23:04:38.183307 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.183314 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.183427 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.183505 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.184230 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.185447 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.187536 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.188183 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.188276 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.188883 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.189012 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.189066 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.189336 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.189444 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 23:04:38.239565 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:38.230513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.24.31.252:39111)
0809 23:04:38.230701 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.230708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.237774 (+ 7066us) server_negotiation.cc:407] Connection header received
0809 23:04:38.238041 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.238047 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.238150 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.238259 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:38.239306 (+ 1047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54516: BlockingRecv error: recv got EOF from 127.0.0.1:54516 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 23:04:38.272481 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.266837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.24.31.253:45045)
0809 23:04:38.267063 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.267077 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.267662 (+ 585us) server_negotiation.cc:407] Connection header received
0809 23:04:38.267980 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.267985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.268051 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.268159 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.268593 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.269395 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.270697 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.271164 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.271332 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.271747 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.271849 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.271964 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.272112 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.272190 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 23:04:38.287173 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:38.278776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54530 (local address 127.24.31.252:39111)
0809 23:04:38.279050 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.279058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.285701 (+ 6643us) server_negotiation.cc:407] Connection header received
0809 23:04:38.285956 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.285961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.286040 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.286152 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:38.286927 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54530: BlockingRecv error: recv got EOF from 127.0.0.1:54530 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
I20260809 23:04:38.306770 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.301115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.24.31.253:45045)
0809 23:04:38.301416 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.301425 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.301943 (+ 518us) server_negotiation.cc:407] Connection header received
0809 23:04:38.302244 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.302248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.302314 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.302418 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.302827 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.303595 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.304915 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.305415 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.305546 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.305948 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.306077 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.306174 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.306320 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.306399 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
W20260809 23:04:38.315526 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:38.313313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54544 (local address 127.24.31.252:39111)
0809 23:04:38.313564 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.313573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.314086 (+ 513us) server_negotiation.cc:407] Connection header received
0809 23:04:38.314295 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.314301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.314394 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.314502 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:38.315253 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54544: BlockingRecv error: recv got EOF from 127.0.0.1:54544 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 23:04:38.340538 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.335011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36172 (local address 127.24.31.253:45045)
0809 23:04:38.335261 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.335270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.335809 (+ 539us) server_negotiation.cc:407] Connection header received
0809 23:04:38.336046 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.336051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.336118 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.336218 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.336674 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.337549 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.338854 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.339258 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.339416 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.339800 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.339907 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.340008 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.340152 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.340243 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 23:04:38.349104 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:38.347252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54552 (local address 127.24.31.252:39111)
0809 23:04:38.347522 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.347527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.347552 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:04:38.347608 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.347612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.347680 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.347786 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:38.348753 (+ 967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54552: BlockingRecv error: recv got EOF from 127.0.0.1:54552 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 23:04:38.390971 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.385463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36178 (local address 127.24.31.253:45045)
0809 23:04:38.385669 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.385678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.386077 (+ 399us) server_negotiation.cc:407] Connection header received
0809 23:04:38.386318 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.386324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.386417 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.386535 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.387021 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.387724 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.389051 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.389612 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.389764 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.390183 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.390299 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.390402 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.390560 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.390641 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 23:04:38.399190 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:38.397059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54556 (local address 127.24.31.252:39111)
0809 23:04:38.397315 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.397322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.397681 (+ 359us) server_negotiation.cc:407] Connection header received
0809 23:04:38.397850 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.397854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.397926 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.398009 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:38.398939 (+ 930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54556: BlockingRecv error: recv got EOF from 127.0.0.1:54556 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 23:04:38.472411 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.467289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36182 (local address 127.24.31.253:45045)
0809 23:04:38.467485 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.467490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.467881 (+ 391us) server_negotiation.cc:407] Connection header received
0809 23:04:38.468059 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.468065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.468177 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.468275 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.468697 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.469559 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.470875 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.471334 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.471466 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.471858 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.471954 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.472022 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.472147 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.472208 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 23:04:38.480072 18359 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:38.478408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54564 (local address 127.24.31.252:39111)
0809 23:04:38.478641 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.478655 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.478842 (+ 187us) server_negotiation.cc:407] Connection header received
0809 23:04:38.478989 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.478993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.479063 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.479144 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:38.479869 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54564: BlockingRecv error: recv got EOF from 127.0.0.1:54564 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 23:04:38.487488 18377 catalog_manager.cc:1259] Loaded cluster ID: f8caf0480e1f4f6791b92b29df53bc17
I20260809 23:04:38.487567 18377 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3: loading cluster ID for follower catalog manager: success
I20260809 23:04:38.490054 18377 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3: acquiring CA information for follower catalog manager: success
I20260809 23:04:38.491727 18377 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 529ad8978ff54c0d98ca4eee4be863c3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:04:38.617283 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.611869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.24.31.253:45045)
0809 23:04:38.612148 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.612155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.612519 (+ 364us) server_negotiation.cc:407] Connection header received
0809 23:04:38.612753 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.612769 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.612839 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.612943 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.613399 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.614164 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.615541 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.615980 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.616121 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.616527 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.616636 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.616735 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.616884 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.616963 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 23:04:38.629246 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.623185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.24.31.252:39111)
0809 23:04:38.623367 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.623375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.623790 (+ 415us) server_negotiation.cc:407] Connection header received
0809 23:04:38.624020 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.624023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.624176 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.624278 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.624720 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.625534 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.627059 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.627428 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.627516 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.628074 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.628593 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.628708 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.628884 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.628975 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:04:38.642729 18359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.635332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.24.31.252:39111)
0809 23:04:38.635541 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.635549 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.636496 (+ 947us) server_negotiation.cc:407] Connection header received
0809 23:04:38.636837 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.636844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.636926 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.636987 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.637779 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.638634 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.640680 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.641316 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.641427 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.641950 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.642079 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.642167 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.642387 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.642457 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 23:04:38.642966 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.636446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.24.31.253:45045)
0809 23:04:38.636635 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.636640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.637199 (+ 559us) server_negotiation.cc:407] Connection header received
0809 23:04:38.637352 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.637358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.637440 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.637526 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.638271 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.639316 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.640912 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.641540 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.641671 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.642178 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.642303 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.642390 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.642625 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.642702 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 23:04:38.643733 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:38.635157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52604 (local address 127.24.31.254:39897)
0809 23:04:38.635763 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:38.635768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:38.636482 (+ 714us) server_negotiation.cc:407] Connection header received
0809 23:04:38.636901 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:38.636907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:38.637085 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:38.637307 (+ 222us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:38.638049 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.639127 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.641178 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:38.641752 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:38.641893 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:38.642470 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:38.642946 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:38.643040 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:38.643394 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 23:04:38.643479 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":387,"threads_started":1}
I20260809 23:04:38.649612 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 18112
I20260809 23:04:38.650928 18184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:38.763441 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 18112
I20260809 23:04:38.788573 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 18197
I20260809 23:04:38.789827 18263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:38.897487 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 18197
I20260809 23:04:38.920836 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 18277
I20260809 23:04:38.922165 18349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:39.030920 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 18277
2026-08-09T23:04:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4633 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:04:39 dist-test-slave-vxj2 krb5kdc[18564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:04:39 dist-test-slave-vxj2 krb5kdc[18564](info): set up 2 sockets
Aug 09 23:04:39 dist-test-slave-vxj2 krb5kdc[18564](info): commencing operation
krb5kdc: starting...
W20260809 23:04:40.226830 24703 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/test-user.keytab.
Aug 09 23:04:40 dist-test-slave-vxj2 krb5kdc[18564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:04:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:04:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.31.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
Entry for principal kudu/127.24.31.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab.
I20260809 23:04:40.517563 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:40399
--webserver_interface=127.24.31.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:40399,127.24.31.253:36021,127.24.31.252:37553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:04:40.856928 18578 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:04:40.857358 18578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:40.857470 18578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:40.857606 18578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:40.864868 18578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:40.864959 18578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:40.865056 18578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:40.865175 18578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:40.865271 18578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:40.875253 18578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:40399,127.24.31.253:36021,127.24.31.252:37553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.254.keytab
--principal=kudu/127.24.31.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.254:40399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.31.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:40.876567 18578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:40.877955 18578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:40.889847 18583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:40.889806 18584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:40.889880 18586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:40.890725 18578 server_base.cc:1034] running on GCE node
I20260809 23:04:40.891525 18578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:40.892932 18578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:40.894207 18578 hybrid_clock.cc:648] HybridClock initialized: now 1786316680894128 us; error 36 us; skew 500 ppm
I20260809 23:04:40.894570 18578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:40 dist-test-slave-vxj2 krb5kdc[18564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316680, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:40.897876 18578 init.cc:377] Logged in from keytab as kudu/127.24.31.254@KRBTEST.COM (short username kudu)
I20260809 23:04:40.899353 18578 webserver.cc:460] Webserver started at http://127.24.31.254:43565/ using document root <none> and password file <none>
I20260809 23:04:40.900067 18578 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:40.900204 18578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:40.900543 18578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:40.903527 18578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/data/instance:
uuid: "268d30de45af46f6b9a0f8e301927665"
format_stamp: "Formatted at 2026-08-09 23:04:40 on dist-test-slave-vxj2"
server_key: "de6bc5a3128874964de98b79558249a5"
server_key_iv: "be12279dfcaae57f15388c911c73cc86"
server_key_version: "encryptionkey@0"
I20260809 23:04:40.904368 18578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/wal/instance:
uuid: "268d30de45af46f6b9a0f8e301927665"
format_stamp: "Formatted at 2026-08-09 23:04:40 on dist-test-slave-vxj2"
server_key: "de6bc5a3128874964de98b79558249a5"
server_key_iv: "be12279dfcaae57f15388c911c73cc86"
server_key_version: "encryptionkey@0"
I20260809 23:04:40.909602 18578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 23:04:40.913054 18593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:40.914552 18578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 23:04:40.914791 18578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/wal
uuid: "268d30de45af46f6b9a0f8e301927665"
format_stamp: "Formatted at 2026-08-09 23:04:40 on dist-test-slave-vxj2"
server_key: "de6bc5a3128874964de98b79558249a5"
server_key_iv: "be12279dfcaae57f15388c911c73cc86"
server_key_version: "encryptionkey@0"
I20260809 23:04:40.915033 18578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:40.933561 18578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:40.937964 18578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:40.938264 18578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:40.957858 18578 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.254:40399
I20260809 23:04:40.957878 18653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.254:40399 every 8 connection(s)
I20260809 23:04:40.959239 18578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/data/info.pb
I20260809 23:04:40.963124 18654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:40.966701 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 18578
I20260809 23:04:40.966928 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-0/wal/instance
I20260809 23:04:40.967388 24703 external_mini_cluster.cc:1428] Setting key f441ef8938a25ebc67c3a1537fa8638f
I20260809 23:04:40.970451 18654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:04:40 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316680, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.31.253@KRBTEST.COM; defaulting to no policy
I20260809 23:04:40.995051 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:40.974050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:40399 (local address 127.0.0.1:46814)
0809 23:04:40.975355 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:40.975397 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:40.977382 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:40.978286 (+ 904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:40.978294 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:40.979063 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:40.979423 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:40.979448 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:40.981197 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:40.981203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:40.982347 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:40.982363 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:40.983420 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:40.983428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:40.983627 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:40.984373 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:40.984399 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:40.986818 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:40.990415 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:40.990430 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:40.990443 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:40.990909 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:40.991390 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:40.991395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:40.991399 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:40.991576 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:40.992418 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:40.992429 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:40.992666 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 23:04:40.992931 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 23:04:40.993603 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1114,"mutex_wait_us":130,"thread_start_us":873,"threads_started":1}
I20260809 23:04:40.995088 18657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:40.974035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46814 (local address 127.24.31.254:40399)
0809 23:04:40.975058 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:40.975074 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:40.975530 (+ 456us) server_negotiation.cc:407] Connection header received
0809 23:04:40.977774 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:40.977780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:40.978028 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:40.978260 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:40.979674 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:40.980941 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:40.982514 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:40.983141 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:40.983375 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:40.987074 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:40.987084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:40.987097 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:40.987127 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:40.990252 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:40.991064 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:40.991078 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:40.991081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:40.991254 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:40.991703 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:40.991721 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:40.991724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:40.992258 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:40.992492 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:40.993082 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 23:04:40.993436 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"spinlock_wait_cycles":45952,"thread_start_us":518,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.253@KRBTEST.COM" created.
I20260809 23:04:40.998380 18654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:04:40.999648 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:40.999097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33696)
0809 23:04:40.999320 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:40.999475 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 23:04:40.999943 18596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
I20260809 23:04:41.001823 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.001350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33706)
0809 23:04:41.001536 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.001631 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 23:04:41.002039 18654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:36021: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab.
Entry for principal kudu/127.24.31.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab.
I20260809 23:04:41.036911 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.253:36021
--webserver_interface=127.24.31.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab
--principal=kudu/127.24.31.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:40399,127.24.31.253:36021,127.24.31.252:37553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:04:41.051424 18654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } attempt: 1
I20260809 23:04:41.052657 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.052174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33710)
0809 23:04:41.052381 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.052522 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 23:04:41.053766 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.053377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33724)
0809 23:04:41.053536 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.053611 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 23:04:41.053953 18654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:36021: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
I20260809 23:04:41.122205 18654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } attempt: 2
I20260809 23:04:41.123373 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.122925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33732)
0809 23:04:41.123127 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.123225 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 23:04:41.124377 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.123990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33740)
0809 23:04:41.124146 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.124247 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 23:04:41.124567 18654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:36021: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
I20260809 23:04:41.215822 18654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } attempt: 3
I20260809 23:04:41.217003 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.216542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33746)
0809 23:04:41.216746 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.216865 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 23:04:41.218056 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.217707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33756)
0809 23:04:41.217864 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.217936 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 23:04:41.218261 18654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:36021: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
I20260809 23:04:41.361487 18654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } attempt: 4
I20260809 23:04:41.362756 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.362209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33768)
0809 23:04:41.362532 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.362624 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260809 23:04:41.363804 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.363428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33776)
0809 23:04:41.363593 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.363683 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 23:04:41.364056 18654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.253:36021: Network error: Client connection negotiation failed: client connection to 127.24.31.253:36021: connect: Connection refused (error 111)
W20260809 23:04:41.369277 18661 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:04:41.369648 18661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:41.369765 18661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:41.369907 18661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:41.377393 18661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:41.377475 18661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:41.377588 18661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:41.377671 18661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:41.377791 18661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:41.389848 18661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:40399,127.24.31.253:36021,127.24.31.252:37553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.253.keytab
--principal=kudu/127.24.31.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.253:36021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.31.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:41.391683 18661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:41.393121 18661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:41.405329 18668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:41.405453 18667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:41.405426 18670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:41.406657 18661 server_base.cc:1034] running on GCE node
I20260809 23:04:41.407485 18661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:41.408912 18661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:41.410156 18661 hybrid_clock.cc:648] HybridClock initialized: now 1786316681410072 us; error 62 us; skew 500 ppm
I20260809 23:04:41.410593 18661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:41 dist-test-slave-vxj2 krb5kdc[18564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316681, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:41.413990 18661 init.cc:377] Logged in from keytab as kudu/127.24.31.253@KRBTEST.COM (short username kudu)
I20260809 23:04:41.415406 18661 webserver.cc:460] Webserver started at http://127.24.31.253:39683/ using document root <none> and password file <none>
I20260809 23:04:41.416028 18661 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:41.416111 18661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:41.416393 18661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:41.419320 18661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/data/instance:
uuid: "e06e1888b86641dd96267ea9839633cd"
format_stamp: "Formatted at 2026-08-09 23:04:41 on dist-test-slave-vxj2"
server_key: "46402746fd177a847728d55293d01086"
server_key_iv: "09d8ec2665f9169448fd01c307bc5e96"
server_key_version: "encryptionkey@0"
I20260809 23:04:41.420066 18661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/wal/instance:
uuid: "e06e1888b86641dd96267ea9839633cd"
format_stamp: "Formatted at 2026-08-09 23:04:41 on dist-test-slave-vxj2"
server_key: "46402746fd177a847728d55293d01086"
server_key_iv: "09d8ec2665f9169448fd01c307bc5e96"
server_key_version: "encryptionkey@0"
I20260809 23:04:41.425989 18661 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 23:04:41.429878 18678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:41.431298 18661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 23:04:41.431527 18661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/wal
uuid: "e06e1888b86641dd96267ea9839633cd"
format_stamp: "Formatted at 2026-08-09 23:04:41 on dist-test-slave-vxj2"
server_key: "46402746fd177a847728d55293d01086"
server_key_iv: "09d8ec2665f9169448fd01c307bc5e96"
server_key_version: "encryptionkey@0"
I20260809 23:04:41.431839 18661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:41.447247 18661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:41.453127 18661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:41.453553 18661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:41.475642 18661 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.253:36021
I20260809 23:04:41.475648 18735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.253:36021 every 8 connection(s)
I20260809 23:04:41.477313 18661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/data/info.pb
I20260809 23:04:41.481081 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:41.486841 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 18661
I20260809 23:04:41.487090 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-1/wal/instance
I20260809 23:04:41.487640 24703 external_mini_cluster.cc:1428] Setting key 6c6a0d6cd73d50ae5d02ff78b9fa3aac
I20260809 23:04:41.488006 18736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:04:41 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316681, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:04:41.511279 18657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.490640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46816 (local address 127.24.31.254:40399)
0809 23:04:41.490908 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:41.490917 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:41.492068 (+ 1151us) server_negotiation.cc:407] Connection header received
0809 23:04:41.493995 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:41.494005 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:41.494124 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:41.494300 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:41.495832 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.497112 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.498678 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.499360 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.499488 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:41.504053 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:41.504060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:41.504064 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:41.504104 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:41.507486 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:41.508408 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:41.508417 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:41.508424 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:41.508579 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:41.509207 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:41.509222 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:41.509224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:41.509553 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:41.509758 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:41.510569 (+ 811us) server_negotiation.cc:299] Negotiation successful
0809 23:04:41.510849 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 23:04:41.512353 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.491042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:40399 (local address 127.0.0.1:46816)
0809 23:04:41.491849 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.491900 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:41.493714 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:41.494519 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:41.494531 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:41.495101 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:41.495630 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:41.495647 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.497336 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.497339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:41.498502 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:41.498514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.499617 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.499626 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:41.499957 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:41.500818 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:41.500860 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:41.503775 (+ 2915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:41.507706 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:41.507717 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:41.507734 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:41.508256 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:41.508719 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:41.508728 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:41.508734 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:41.508951 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:41.509713 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:41.509723 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:41.510122 (+ 399us) client_negotiation.cc:769] Sending connection context
0809 23:04:41.510456 (+ 334us) client_negotiation.cc:240] Negotiation successful
0809 23:04:41.511029 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":335,"threads_started":1}
I20260809 23:04:41.514459 18736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.31.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.31.252@KRBTEST.COM" created.
Aug 09 23:04:41 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316681, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
I20260809 23:04:41.532070 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.515295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33790)
0809 23:04:41.515466 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.515487 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:41.515589 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:41.516837 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:41.516844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:41.517240 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:41.517382 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:41.517390 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.519117 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.519122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:41.520448 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:41.520463 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.521754 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.521759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:41.521907 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:41.522479 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:41.522495 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:41.524864 (+ 2369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:41.529457 (+ 4593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:41.529461 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:41.529464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:41.529832 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:41.530246 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:41.530250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:41.530252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:41.530393 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:41.531134 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:41.531138 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:41.531255 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 23:04:41.531372 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 23:04:41.531569 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 23:04:41.532091 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.515585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33790 (local address 127.24.31.253:36021)
0809 23:04:41.516371 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:41.516376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:41.516400 (+ 24us) server_negotiation.cc:407] Connection header received
0809 23:04:41.516439 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:41.516444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:41.516662 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:41.516774 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:41.517563 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.518938 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.520608 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.521557 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.521701 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:41.525095 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:41.525110 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:41.525129 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:41.525196 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:41.529305 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:41.529969 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:41.529981 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:41.529987 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:41.530135 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:41.530499 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:41.530506 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:41.530512 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:41.530997 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:41.531209 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:41.531567 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 23:04:41.531798 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":459,"threads_started":1}
I20260809 23:04:41.534911 18736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:04:41.536245 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.535662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57412)
0809 23:04:41.535908 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.536040 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 23:04:41.536597 18679 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
I20260809 23:04:41.538385 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.537966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57414)
0809 23:04:41.538146 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.538250 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 23:04:41.538622 18736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:37553: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.31.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab.
Entry for principal kudu/127.24.31.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab.
I20260809 23:04:41.558837 24703 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
/tmp/dist-test-taskHiIJT1/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.252:37553
--webserver_interface=127.24.31.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab
--principal=kudu/127.24.31.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.31.212:38559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.31.254:40399,127.24.31.253:36021,127.24.31.252:37553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:04:41.587988 18736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } attempt: 1
I20260809 23:04:41.589305 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.588742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57428)
0809 23:04:41.588971 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.589107 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 23:04:41.590422 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.590043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57436)
0809 23:04:41.590204 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.590293 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 23:04:41.590621 18736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:37553: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
I20260809 23:04:41.658880 18736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } attempt: 2
I20260809 23:04:41.660173 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.659625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57444)
0809 23:04:41.659868 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.659983 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 23:04:41.661401 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.660938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57456)
0809 23:04:41.661126 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.661259 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 23:04:41.661669 18736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:37553: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
I20260809 23:04:41.663358 18654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } attempt: 5
Aug 09 23:04:41 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316680, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
I20260809 23:04:41.677767 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.664097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.24.31.253:36021)
0809 23:04:41.664293 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:41.664297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:41.664341 (+ 44us) server_negotiation.cc:407] Connection header received
0809 23:04:41.664546 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:41.664550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:41.664634 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:41.664767 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:41.665786 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.666525 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.668200 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.668679 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.668833 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:41.672433 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:41.672438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:41.672441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:41.672473 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:41.675202 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:41.675844 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:41.675851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:41.675855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:41.675975 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:41.676365 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:41.676371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:41.676375 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:41.676651 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:41.676820 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:41.677222 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 23:04:41.677427 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 23:04:41.677760 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.664013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33804)
0809 23:04:41.664218 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.664257 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:41.664409 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:41.664803 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:41.664808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:41.665333 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:41.665581 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:41.665593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.666674 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.666679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:41.668025 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:41.668041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:41.668897 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:41.668903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:41.669100 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:41.669748 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:41.669779 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:41.672208 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:41.675342 (+ 3134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:41.675348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:41.675353 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:41.675719 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:41.676092 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:41.676098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:41.676101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:41.676254 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:41.676783 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:41.676788 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:41.676946 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 23:04:41.677096 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 23:04:41.677409 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 23:04:41.678799 18654 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:04:41.680011 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.679562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57470)
0809 23:04:41.679743 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.679844 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 23:04:41.681152 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.680685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57478)
0809 23:04:41.680863 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.680976 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 23:04:41.681406 18654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:37553: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
I20260809 23:04:41.732641 18654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } attempt: 1
I20260809 23:04:41.733892 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.733378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57486)
0809 23:04:41.733580 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.733685 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 23:04:41.734967 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.734593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57488)
0809 23:04:41.734742 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.734837 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 23:04:41.735246 18654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:37553: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
I20260809 23:04:41.752909 18736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } attempt: 3
I20260809 23:04:41.754223 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.753673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57504)
0809 23:04:41.753913 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.754032 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 23:04:41.755399 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.754916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57512)
0809 23:04:41.755161 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.755256 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260809 23:04:41.755697 18736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:37553: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
I20260809 23:04:41.803504 18654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } attempt: 2
I20260809 23:04:41.804909 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.804291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57514)
0809 23:04:41.804560 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.804687 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 23:04:41.806106 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.805666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57518)
0809 23:04:41.805865 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.805961 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 23:04:41.806354 18654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:37553: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
I20260809 23:04:41.898979 18736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } attempt: 4
I20260809 23:04:41.900233 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.899743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57530)
0809 23:04:41.899975 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.900078 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 23:04:41.901386 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.901008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57540)
0809 23:04:41.901144 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.901284 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 23:04:41.901662 18736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:37553: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
W20260809 23:04:41.902063 18744 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:04:41.902472 18744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:04:41.902595 18744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:04:41.902737 18744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:04:41.910264 18744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:04:41.910386 18744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:04:41.910506 18744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:04:41.910593 18744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:04:41.910709 18744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:04:41.912606 18654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } attempt: 3
I20260809 23:04:41.913870 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.913317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57544)
0809 23:04:41.913555 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.913660 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 23:04:41.914945 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:41.914518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57556)
0809 23:04:41.914730 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:41.914824 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260809 23:04:41.915212 18654 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.31.252:37553: Network error: Client connection negotiation failed: client connection to 127.24.31.252:37553: connect: Connection refused (error 111)
I20260809 23:04:41.921823 18744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.31.212:38559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.31.254:40399,127.24.31.253:36021,127.24.31.252:37553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/krb5kdc/kudu_127.24.31.252.keytab
--principal=kudu/127.24.31.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.31.252:37553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.31.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 21:55:06 UTC on ccycloud.quasar-enrzoz.root.comops.site
build id 4576815
ASAN enabled
I20260809 23:04:41.923038 18744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:04:41.924326 18744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:04:41.936524 18752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:41.936497 18750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:04:41.937026 18749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:04:41.937422 18744 server_base.cc:1034] running on GCE node
I20260809 23:04:41.938133 18744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:04:41.939603 18744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:04:41.940927 18744 hybrid_clock.cc:648] HybridClock initialized: now 1786316681940839 us; error 46 us; skew 500 ppm
I20260809 23:04:41.941323 18744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 23:04:41 dist-test-slave-vxj2 krb5kdc[18564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316681, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:04:41.944625 18744 init.cc:377] Logged in from keytab as kudu/127.24.31.252@KRBTEST.COM (short username kudu)
I20260809 23:04:41.946067 18744 webserver.cc:460] Webserver started at http://127.24.31.252:38241/ using document root <none> and password file <none>
I20260809 23:04:41.946723 18744 fs_manager.cc:359] Metadata directory not provided
I20260809 23:04:41.946818 18744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:04:41.947144 18744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:04:41.950116 18744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/data/instance:
uuid: "10779e0cfb7348f29ec17c47039ba147"
format_stamp: "Formatted at 2026-08-09 23:04:41 on dist-test-slave-vxj2"
server_key: "ba1803328cfa76e1c4c98c7f5d8dedee"
server_key_iv: "e1d174defecb4703058cb63ed0152a7b"
server_key_version: "encryptionkey@0"
I20260809 23:04:41.950976 18744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/wal/instance:
uuid: "10779e0cfb7348f29ec17c47039ba147"
format_stamp: "Formatted at 2026-08-09 23:04:41 on dist-test-slave-vxj2"
server_key: "ba1803328cfa76e1c4c98c7f5d8dedee"
server_key_iv: "e1d174defecb4703058cb63ed0152a7b"
server_key_version: "encryptionkey@0"
I20260809 23:04:41.955991 18744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 23:04:41.959527 18760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:04:41.960857 18744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 23:04:41.961046 18744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/data,/tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/wal
uuid: "10779e0cfb7348f29ec17c47039ba147"
format_stamp: "Formatted at 2026-08-09 23:04:41 on dist-test-slave-vxj2"
server_key: "ba1803328cfa76e1c4c98c7f5d8dedee"
server_key_iv: "e1d174defecb4703058cb63ed0152a7b"
server_key_version: "encryptionkey@0"
I20260809 23:04:41.961338 18744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:04:42.005390 18744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:04:42.010318 18744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:04:42.010644 18744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:04:42.031015 18744 rpc_server.cc:307] RPC server started. Bound to: 127.24.31.252:37553
I20260809 23:04:42.031023 18815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.31.252:37553 every 8 connection(s)
I20260809 23:04:42.032444 18744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/data/info.pb
I20260809 23:04:42.036214 18816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:04:42.039000 24703 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu as pid 18744
I20260809 23:04:42.039187 24703 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHiIJT1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316311377261-24703-0/minicluster-data/master-2/wal/instance
I20260809 23:04:42.039723 24703 external_mini_cluster.cc:1428] Setting key 90322918a6d05ccbeee3a65577a7c7c4
I20260809 23:04:42.044023 18816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:04:42 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
Aug 09 23:04:42 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316681, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:04:42.063246 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.046719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.24.31.254:40399)
0809 23:04:42.048211 (+ 1492us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.048215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.048231 (+ 16us) server_negotiation.cc:407] Connection header received
0809 23:04:42.048278 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.048282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.048357 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.048441 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.049918 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.050937 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.052663 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.053089 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.053309 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.057356 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.057362 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.057365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.057400 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.060138 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.060885 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.060891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.060895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.061037 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.061520 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.061528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.061535 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.061807 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.061988 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.062503 (+ 515us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.062768 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1392,"thread_start_us":1303,"threads_started":1}
I20260809 23:04:42.063899 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.041647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.24.31.252:37553)
0809 23:04:42.042482 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.042493 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.042535 (+ 42us) server_negotiation.cc:407] Connection header received
0809 23:04:42.044392 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.044417 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.044856 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.045338 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.047290 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.049028 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.050683 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.051646 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.051820 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.056348 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.056378 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.056400 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.056449 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.059646 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.060369 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.060381 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.060383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.060544 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.060953 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.060960 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.060963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.061549 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.061853 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.062278 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.062729 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":404,"threads_started":1}
I20260809 23:04:42.063984 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.046665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.254:40399 (local address 127.0.0.1:46830)
0809 23:04:42.047470 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:42.047535 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:42.047723 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:42.048556 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:42.048575 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:42.049491 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:42.049749 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.049757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.051097 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.051102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.052508 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.052519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.053420 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.053429 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.053594 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.054207 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:42.054235 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:42.057136 (+ 2901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:42.060293 (+ 3157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.060305 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.060317 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.060744 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.061191 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.061197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.061200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.061397 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.061968 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:42.061976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:42.062183 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 23:04:42.062373 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 23:04:42.062888 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":119,"thread_start_us":297,"threads_started":1}
I20260809 23:04:42.066303 18816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:04:42 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316681, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
Aug 09 23:04:42 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.254@KRBTEST.COM
I20260809 23:04:42.081739 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.067157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.253:36021 (local address 127.0.0.1:33808)
0809 23:04:42.067484 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:42.067502 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:42.067599 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:42.067972 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:42.067978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:42.068454 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:42.068683 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.068695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.070219 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.070224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.071154 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.071166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.072164 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.072171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.072428 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.073022 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:42.073049 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:42.075635 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:42.079196 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.079202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.079208 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.079756 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.080133 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.080137 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.080140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.080335 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.080820 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:42.080825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:42.081035 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 23:04:42.081257 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 23:04:42.081490 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260809 23:04:42.082072 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.067148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33808 (local address 127.24.31.253:36021)
0809 23:04:42.067362 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.067369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.067582 (+ 213us) server_negotiation.cc:407] Connection header received
0809 23:04:42.067734 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.067739 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.067824 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.067934 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.068884 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.070048 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.071303 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.072025 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.072126 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.075873 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.075881 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.075885 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.075921 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.079055 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.079880 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.079884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.079887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.080010 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.080447 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.080450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.080451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.080685 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.080811 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.081368 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.081597 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 23:04:42.083119 18816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:04:42.084381 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.067084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46844 (local address 127.24.31.254:40399)
0809 23:04:42.067373 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.067378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.067398 (+ 20us) server_negotiation.cc:407] Connection header received
0809 23:04:42.067611 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.067618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.067746 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.067854 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.069476 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.070311 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.071909 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.072453 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.072595 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.077630 (+ 5035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.077636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.077639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.077670 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.080934 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.081972 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.081983 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.081989 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.082155 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.082646 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.082651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.082655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.083071 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.083383 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.083757 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.084013 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
Aug 09 23:04:42 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316681, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.252@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
I20260809 23:04:42.092427 18654 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } attempt: 4
Aug 09 23:04:42 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316680, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.31.253@KRBTEST.COM
I20260809 23:04:42.100315 18820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.083976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57564)
0809 23:04:42.084174 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:42.084188 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:42.084306 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:42.084731 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:42.084734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:42.085055 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:42.085236 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.085245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.086656 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.086662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.088085 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.088102 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.089070 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.089074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.089354 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.089925 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:42.089948 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:42.092684 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:42.097554 (+ 4870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.097559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.097561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.098046 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.098573 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.098578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.098582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.098728 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.099408 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:42.099413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:42.099602 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 23:04:42.099787 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 23:04:42.100031 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":35}
Aug 09 23:04:42 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316680, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.254@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
I20260809 23:04:42.102016 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.083908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.24.31.252:37553)
0809 23:04:42.084078 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.084084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.084264 (+ 180us) server_negotiation.cc:407] Connection header received
0809 23:04:42.084420 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.084427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.084507 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.084594 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.085386 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.086527 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.088229 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.088921 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.089058 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.092923 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.092929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.092934 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.092971 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.097427 (+ 4456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.098180 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.098189 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.098196 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.098388 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.098843 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.098851 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.098855 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.099259 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.099446 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.101351 (+ 1905us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.101606 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":61}
I20260809 23:04:42.104365 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.086145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33822 (local address 127.24.31.253:36021)
0809 23:04:42.086305 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.086311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.086328 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:04:42.086374 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.086379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.086455 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.086586 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.088150 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.089346 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.091123 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.091827 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.091983 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.097543 (+ 5560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.097550 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.097556 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.097604 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.100917 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.101826 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.101833 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.101837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.102307 (+ 470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.102842 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.102850 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.102856 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.103219 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.103477 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.103792 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.104019 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:04:42.108155 18816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147: Bootstrap starting.
I20260809 23:04:42.109531 24703 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:04:42.111634 18816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:42.112924 18816 log.cc:824] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:42.115980 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.093060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57576)
0809 23:04:42.093265 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:42.093284 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:42.093404 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:42.094032 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:42.094040 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:42.094634 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:42.094795 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.094805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.096273 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.096282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.097407 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.097442 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.099059 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.099063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.099276 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.099853 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:42.099876 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:42.102958 (+ 3082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:42.112530 (+ 9572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.112536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.112539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.113229 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.113854 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.113865 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.113869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.114063 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.114847 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:42.114852 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:42.115055 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 23:04:42.115217 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 23:04:42.115547 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 23:04:42.116369 18816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147: No bootstrap required, opened a new log
I20260809 23:04:42.116534 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.093101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57576 (local address 127.24.31.252:37553)
0809 23:04:42.093757 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.093761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.093779 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:04:42.093815 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.093818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.093865 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.094006 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.094973 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.096132 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.097554 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.098260 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.098416 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.103219 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.103226 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.103230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.103278 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.112354 (+ 9076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.113402 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.113416 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.113420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.113639 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.114256 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.114261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.114265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.114703 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.114861 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.115365 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.115611 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":365,"threads_started":1}
I20260809 23:04:42.112201 18625 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "268d30de45af46f6b9a0f8e301927665"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46844
I20260809 23:04:42.121722 18654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665: Bootstrap starting.
I20260809 23:04:42.125013 18654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:42.126155 18654 log.cc:824] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665: Log is configured to *not* fsync() on all Append() calls
I20260809 23:04:42.126650 18816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } }
I20260809 23:04:42.127087 18816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10779e0cfb7348f29ec17c47039ba147, State: Initialized, Role: FOLLOWER
I20260809 23:04:42.127624 18816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } }
I20260809 23:04:42.128683 18826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.129119 18826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:42.129609 18654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665: No bootstrap required, opened a new log
I20260809 23:04:42.129977 18816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:42.137842 18654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } }
I20260809 23:04:42.138242 18654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 268d30de45af46f6b9a0f8e301927665, State: Initialized, Role: FOLLOWER
I20260809 23:04:42.138733 18654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } }
W20260809 23:04:42.140854 18838 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:04:42.141072 18838 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:04:42.145671 18840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.146062 18840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:42.147810 18654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:42.147938 18625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 23:04:42.148164 18625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:42.137964 18744 master_runner.cc:186] Error getting master registration for 127.24.31.254:40399: OK, instance_id { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" instance_seqno: 1786316680952157 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:04:42.151932 18744 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:04:42.152261 18625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } }
I20260809 23:04:42.155612 18790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "268d30de45af46f6b9a0f8e301927665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "10779e0cfb7348f29ec17c47039ba147"
I20260809 23:04:42.156154 18790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:42.160178 18790 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 268d30de45af46f6b9a0f8e301927665 in term 1.
I20260809 23:04:42.160984 18594 leader_election.cc:302] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10779e0cfb7348f29ec17c47039ba147, 268d30de45af46f6b9a0f8e301927665; no voters:
I20260809 23:04:42.161600 18840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:04:42.161700 18625 leader_election.cc:288] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [CANDIDATE]: Term 1 election: Requested vote from peers e06e1888b86641dd96267ea9839633cd (127.24.31.253:36021), 10779e0cfb7348f29ec17c47039ba147 (127.24.31.252:37553)
I20260809 23:04:42.176612 18840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [term 1 LEADER]: Becoming Leader. State: Replica: 268d30de45af46f6b9a0f8e301927665, State: Running, Role: LEADER
I20260809 23:04:42.178211 18840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } }
W20260809 23:04:42.191220 18600 tablet.cc:2369] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:04:42.192756 18849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 268d30de45af46f6b9a0f8e301927665. Latest consensus state: current_term: 1 leader_uuid: "268d30de45af46f6b9a0f8e301927665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.193869 18849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:42.200855 18736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } attempt: 5
I20260809 23:04:42.203819 18859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 23:04:42 dist-test-slave-vxj2 krb5kdc[18564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316681, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.31.253@KRBTEST.COM for kudu/127.24.31.252@KRBTEST.COM
I20260809 23:04:42.224879 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.201513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.31.252:37553 (local address 127.0.0.1:57594)
0809 23:04:42.201850 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 23:04:42.201875 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 23:04:42.202021 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:04:42.203106 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:04:42.203112 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:04:42.203674 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:04:42.203901 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.203928 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.205694 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.205701 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.207285 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:04:42.207301 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.208764 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.208771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:04:42.209030 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.209775 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:04:42.209802 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:04:42.213365 (+ 3563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:04:42.219371 (+ 6006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.219378 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.219383 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.220003 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.221897 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:04:42.221903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:04:42.221919 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:04:42.222099 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:04:42.223726 (+ 1627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:04:42.223733 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:04:42.223925 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 23:04:42.224095 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 23:04:42.224425 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260809 23:04:42.227293 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.202387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57594 (local address 127.24.31.252:37553)
0809 23:04:42.202582 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.202589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.202609 (+ 20us) server_negotiation.cc:407] Connection header received
0809 23:04:42.202675 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.202681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.202765 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.202876 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.204185 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.205487 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.207489 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.208359 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.208495 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.215064 (+ 6569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.215077 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.215086 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.215139 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.219201 (+ 4062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.221269 (+ 2068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.221275 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.221279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.221447 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.222676 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.222682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.222685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.223176 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.223348 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.226614 (+ 3266us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.226871 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 23:04:42.236853 18736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd: Bootstrap starting.
I20260809 23:04:42.244019 18736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd: Neither blocks nor log segments found. Creating new log.
I20260809 23:04:42.248381 18736 log.cc:824] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd: Log is configured to *not* fsync() on all Append() calls
W20260809 23:04:42.263288 18768 tablet.cc:2369] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:04:42.268191 18736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd: No bootstrap required, opened a new log
I20260809 23:04:42.285326 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.200167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57580 (local address 127.24.31.252:37553)
0809 23:04:42.200379 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.200388 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.253769 (+ 53381us) server_negotiation.cc:407] Connection header received
0809 23:04:42.254077 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.254083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.254191 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.254275 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.256733 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.257920 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.264739 (+ 6819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.265498 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.265615 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.277378 (+ 11763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.277387 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.277394 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.277462 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.281356 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.282259 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.282265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.282268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.282457 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.282962 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.282971 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.282975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.283437 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.283686 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.284581 (+ 895us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.284840 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 23:04:42.287313 18859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:04:42.306340 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.197723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.24.31.253:36021)
0809 23:04:42.197958 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.197966 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.250272 (+ 52306us) server_negotiation.cc:407] Connection header received
0809 23:04:42.250822 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.250830 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.250947 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.251059 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.254482 (+ 3423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.255793 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.273311 (+ 17518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.274098 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.274213 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.289056 (+ 14843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.289064 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.289070 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.289114 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.293087 (+ 3973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.294069 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.294076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.294079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.294246 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.301227 (+ 6981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.301233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.301238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.301665 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.301935 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.305640 (+ 3705us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.305922 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 23:04:42.315272 18790 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [term 1 FOLLOWER]: Refusing update from remote peer 268d30de45af46f6b9a0f8e301927665: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:04:42.316223 18849 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:04:42.318459 18821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.245552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.24.31.254:40399)
0809 23:04:42.245791 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.245800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.259030 (+ 13230us) server_negotiation.cc:407] Connection header received
0809 23:04:42.259313 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.259320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.259407 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.259486 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.262178 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.263539 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.281302 (+ 17763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.282111 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.282235 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.307585 (+ 25350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:04:42.307596 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:04:42.307605 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:04:42.307690 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:04:42.314420 (+ 6730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.315555 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.315562 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.315566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.315730 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:04:42.316234 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:04:42.316242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:04:42.316245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:04:42.316821 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:04:42.317138 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.317773 (+ 635us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.318034 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 23:04:42.312712 18578 master_runner.cc:186] Error getting master registration for 127.24.31.253:36021: OK, instance_id { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" instance_seqno: 1786316681469362 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:04:42.330353 18578 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:04:42.338905 18826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 268d30de45af46f6b9a0f8e301927665. Latest consensus state: current_term: 1 leader_uuid: "268d30de45af46f6b9a0f8e301927665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.339244 18826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:42.340306 18871 mvcc.cc:205] Tried to move back new op lower bound from 7316753130750156800 to 7316753130220793856. Current Snapshot: MvccSnapshot[applied={T|T < 7316753130750156800}]
I20260809 23:04:42.340648 18736 raft_consensus.cc:348] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } }
I20260809 23:04:42.341109 18736 raft_consensus.cc:729] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06e1888b86641dd96267ea9839633cd, State: Initialized, Role: FOLLOWER
I20260809 23:04:42.341919 18872 mvcc.cc:205] Tried to move back new op lower bound from 7316753130750156800 to 7316753130220793856. Current Snapshot: MvccSnapshot[applied={T|T < 7316753130750156800}]
I20260809 23:04:42.341714 18736 consensus_queue.cc:260] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } }
I20260809 23:04:42.348513 18826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "268d30de45af46f6b9a0f8e301927665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.348816 18826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:42.350453 18840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "268d30de45af46f6b9a0f8e301927665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.350800 18840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:42.351878 18736 sys_catalog.cc:564] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:04:42.352458 18859 catalog_manager.cc:1347] Generated new cluster ID: a80a43360bab45b683f28b971ab6500d
I20260809 23:04:42.352209 18707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "268d30de45af46f6b9a0f8e301927665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e06e1888b86641dd96267ea9839633cd"
I20260809 23:04:42.352545 18859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:04:42.352708 18707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:04:42.356525 18707 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 268d30de45af46f6b9a0f8e301927665 in term 1.
I20260809 23:04:42.357296 18706 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [term 1 FOLLOWER]: Refusing update from remote peer 268d30de45af46f6b9a0f8e301927665: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:04:42.358888 18849 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:04:42.349692 18876 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.363026 18876 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:42.390189 18859 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:04:42.392485 18859 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:04:42.394551 18876 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 268d30de45af46f6b9a0f8e301927665. Latest consensus state: current_term: 1 leader_uuid: "268d30de45af46f6b9a0f8e301927665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.394840 18876 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:42.395377 18878 sys_catalog.cc:455] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "268d30de45af46f6b9a0f8e301927665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.395736 18878 sys_catalog.cc:458] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:04:42.397013 18849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "268d30de45af46f6b9a0f8e301927665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "268d30de45af46f6b9a0f8e301927665" member_type: VOTER last_known_addr { host: "127.24.31.254" port: 40399 } } peers { permanent_uuid: "e06e1888b86641dd96267ea9839633cd" member_type: VOTER last_known_addr { host: "127.24.31.253" port: 36021 } } peers { permanent_uuid: "10779e0cfb7348f29ec17c47039ba147" member_type: VOTER last_known_addr { host: "127.24.31.252" port: 37553 } } }
I20260809 23:04:42.397289 18849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665 [sys.catalog]: This master's current role is: LEADER
I20260809 23:04:42.413820 18859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 268d30de45af46f6b9a0f8e301927665: Generated new TSK 0
I20260809 23:04:42.414952 18859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:04:42.427822 18891 catalog_manager.cc:1259] Loaded cluster ID: a80a43360bab45b683f28b971ab6500d
I20260809 23:04:42.428006 18891 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd: loading cluster ID for follower catalog manager: success
I20260809 23:04:42.431871 18891 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd: acquiring CA information for follower catalog manager: success
I20260809 23:04:42.434489 18891 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e06e1888b86641dd96267ea9839633cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 23:04:42.558288 18741 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:42.547090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.24.31.253:36021)
0809 23:04:42.547326 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.547333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.556230 (+ 8897us) server_negotiation.cc:407] Connection header received
0809 23:04:42.556542 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.556550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.556649 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.556741 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.558027 (+ 1286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33852: BlockingRecv error: recv got EOF from 127.0.0.1:33852 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 23:04:42.560616 18822 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:42.549212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57602 (local address 127.24.31.252:37553)
0809 23:04:42.549407 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.549413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.558911 (+ 9498us) server_negotiation.cc:407] Connection header received
0809 23:04:42.559149 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.559155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.559235 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.559318 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.560331 (+ 1013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57602: BlockingRecv error: recv got EOF from 127.0.0.1:57602 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 23:04:42.563017 18821 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:42.556861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46856 (local address 127.24.31.254:40399)
0809 23:04:42.557171 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.557178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.561395 (+ 4217us) server_negotiation.cc:407] Connection header received
0809 23:04:42.561647 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.561653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.561739 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.561812 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.562783 (+ 971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46856: BlockingRecv error: recv got EOF from 127.0.0.1:46856 (error 108)
Metrics: {"server-negotiator.queue_time_us":154}
I20260809 23:04:42.609720 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.593587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33864 (local address 127.24.31.253:36021)
0809 23:04:42.593773 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.593782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.601128 (+ 7346us) server_negotiation.cc:407] Connection header received
0809 23:04:42.601722 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.601734 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.601949 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.602086 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:42.602935 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.604079 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.606443 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.607297 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.607400 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.608157 (+ 757us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:42.608859 (+ 702us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:42.608981 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.609278 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.609380 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 23:04:42.655131 18822 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:42.643901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57612 (local address 127.24.31.252:37553)
0809 23:04:42.644110 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.644117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.649123 (+ 5006us) server_negotiation.cc:407] Connection header received
0809 23:04:42.653436 (+ 4313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.653442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.653593 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.653710 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.654878 (+ 1168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57612: BlockingRecv error: recv got EOF from 127.0.0.1:57612 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 23:04:42.704952 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.687242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33870 (local address 127.24.31.253:36021)
0809 23:04:42.687512 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.687519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.697790 (+ 10271us) server_negotiation.cc:407] Connection header received
0809 23:04:42.698038 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.698045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.698175 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.698305 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:42.699143 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.700335 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.702459 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.703182 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.703311 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.703996 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:42.704166 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:42.704251 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.704518 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.704617 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260809 23:04:42.762128 18822 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:42.749923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57626 (local address 127.24.31.252:37553)
0809 23:04:42.750109 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.750116 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.759909 (+ 9793us) server_negotiation.cc:407] Connection header received
0809 23:04:42.760522 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.760529 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.760633 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.760727 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.761874 (+ 1147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57626: BlockingRecv error: recv got EOF from 127.0.0.1:57626 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 23:04:42.813019 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.797845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33880 (local address 127.24.31.253:36021)
0809 23:04:42.798094 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.798101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.805687 (+ 7586us) server_negotiation.cc:407] Connection header received
0809 23:04:42.806212 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.806218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.806337 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.806417 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:42.807162 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.808387 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.810567 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.811299 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.811407 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.812065 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:42.812215 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:42.812296 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.812573 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.812662 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
W20260809 23:04:42.876251 18822 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:42.863084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57632 (local address 127.24.31.252:37553)
0809 23:04:42.863278 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.863284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.874028 (+ 10744us) server_negotiation.cc:407] Connection header received
0809 23:04:42.874579 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.874586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.874684 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.874760 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.875970 (+ 1210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57632: BlockingRecv error: recv got EOF from 127.0.0.1:57632 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 23:04:42.930338 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.923373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33896 (local address 127.24.31.253:36021)
0809 23:04:42.923568 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.923574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.925401 (+ 1827us) server_negotiation.cc:407] Connection header received
0809 23:04:42.925650 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.925655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.925736 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.925854 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:42.926304 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.927113 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.928429 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.928873 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.929009 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.929501 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:42.929610 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:42.929708 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.929861 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.929958 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 23:04:42.941223 18822 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:42.939369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57646 (local address 127.24.31.252:37553)
0809 23:04:42.939627 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.939633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.939696 (+ 63us) server_negotiation.cc:407] Connection header received
0809 23:04:42.939997 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.940003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.940106 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.940237 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.940988 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57646: BlockingRecv error: recv got EOF from 127.0.0.1:57646 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
I20260809 23:04:42.969269 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:42.953346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.24.31.253:36021)
0809 23:04:42.953617 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.953623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.962229 (+ 8606us) server_negotiation.cc:407] Connection header received
0809 23:04:42.962701 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.962707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.962801 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.962860 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:42.963461 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.964755 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.966983 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:42.967650 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:42.967750 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:42.968341 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:42.968530 (+ 189us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:42.968615 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:42.968826 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 23:04:42.968909 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
W20260809 23:04:43.000142 18822 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:42.997669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.24.31.252:37553)
0809 23:04:42.997926 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:42.997936 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:42.998342 (+ 406us) server_negotiation.cc:407] Connection header received
0809 23:04:42.998570 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:42.998575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:42.998687 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:42.998848 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:42.999893 (+ 1045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57650: BlockingRecv error: recv got EOF from 127.0.0.1:57650 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 23:04:43.024906 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:43.019445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33920 (local address 127.24.31.253:36021)
0809 23:04:43.019699 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:43.019709 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:43.020296 (+ 587us) server_negotiation.cc:407] Connection header received
0809 23:04:43.020528 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:43.020533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:43.020599 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:43.020697 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:43.021120 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.021934 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.023189 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.023605 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.023781 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:43.024184 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:43.024295 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:43.024400 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:43.024546 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 23:04:43.024632 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 23:04:43.034845 18822 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:43.030521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.24.31.252:37553)
0809 23:04:43.030869 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:43.030879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:43.033409 (+ 2530us) server_negotiation.cc:407] Connection header received
0809 23:04:43.033547 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:43.033552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:43.033630 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:43.033720 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:43.034627 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57664: BlockingRecv error: recv got EOF from 127.0.0.1:57664 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 23:04:43.077226 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:43.071325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.24.31.253:36021)
0809 23:04:43.071613 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:43.071621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:43.071964 (+ 343us) server_negotiation.cc:407] Connection header received
0809 23:04:43.072220 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:43.072225 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:43.072320 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:43.072438 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:43.072929 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.073708 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.075212 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.075771 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.075932 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:43.076392 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:43.076498 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:43.076610 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:43.076783 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 23:04:43.076869 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
W20260809 23:04:43.084654 18822 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:04:43.082719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.24.31.252:37553)
0809 23:04:43.083046 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:43.083052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:43.083337 (+ 285us) server_negotiation.cc:407] Connection header received
0809 23:04:43.083554 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:43.083558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:43.083627 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:43.083730 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:04:43.084487 (+ 757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57680: BlockingRecv error: recv got EOF from 127.0.0.1:57680 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
I20260809 23:04:43.143146 18838 catalog_manager.cc:1259] Loaded cluster ID: a80a43360bab45b683f28b971ab6500d
I20260809 23:04:43.143260 18838 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147: loading cluster ID for follower catalog manager: success
I20260809 23:04:43.145867 18838 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147: acquiring CA information for follower catalog manager: success
I20260809 23:04:43.147692 18838 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 10779e0cfb7348f29ec17c47039ba147: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:04:43.157729 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:43.152522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.24.31.253:36021)
0809 23:04:43.152811 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:43.152820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:43.153208 (+ 388us) server_negotiation.cc:407] Connection header received
0809 23:04:43.153380 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:43.153384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:43.153451 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:43.153548 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:43.153943 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.154686 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.156006 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.156417 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.156511 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:43.156923 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:43.157026 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:43.157092 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:43.157377 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 23:04:43.157439 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 23:04:43.168957 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:43.163362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57686 (local address 127.24.31.252:37553)
0809 23:04:43.163615 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:43.163621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:43.163835 (+ 214us) server_negotiation.cc:407] Connection header received
0809 23:04:43.163969 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:43.163973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:43.164122 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:43.164216 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:43.164652 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.165465 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.166782 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.167203 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.167343 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:43.167843 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:43.168343 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:43.168503 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:43.168585 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 23:04:43.168667 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 23:04:43.192934 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:43.174812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57692 (local address 127.24.31.252:37553)
0809 23:04:43.175786 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:43.175794 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:43.186853 (+ 11059us) server_negotiation.cc:407] Connection header received
0809 23:04:43.187069 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:43.187077 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:43.187160 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:43.187274 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:43.187961 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.189028 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.190798 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.191441 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.191524 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:43.192073 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:43.192204 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:43.192300 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:43.192485 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 23:04:43.192600 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787}
I20260809 23:04:43.193439 18741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:43.174849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.24.31.253:36021)
0809 23:04:43.175123 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:43.175132 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:43.187561 (+ 12429us) server_negotiation.cc:407] Connection header received
0809 23:04:43.187823 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:43.187829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:43.187916 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:43.188007 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:43.188555 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.189504 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.191329 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.192019 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.192138 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:43.192597 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:43.192752 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:43.192815 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:43.193027 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 23:04:43.193105 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260809 23:04:43.193661 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:04:43.174542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46862 (local address 127.24.31.254:40399)
0809 23:04:43.185886 (+ 11344us) server_negotiation.cc:206] Beginning negotiation
0809 23:04:43.185895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:04:43.186366 (+ 471us) server_negotiation.cc:407] Connection header received
0809 23:04:43.186692 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:04:43.186706 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:04:43.186911 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:04:43.187125 (+ 214us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:04:43.187704 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.188665 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.190743 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:04:43.191363 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:04:43.191499 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:04:43.192066 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:04:43.192587 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:04:43.192694 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 23:04:43.193208 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 23:04:43.193298 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11163,"thread_start_us":11354,"threads_started":1}
I20260809 23:04:43.199115 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 18578
I20260809 23:04:43.200069 18649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:43.311481 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 18578
I20260809 23:04:43.335386 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 18661
I20260809 23:04:43.336588 18731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:43.447496 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 18661
I20260809 23:04:43.470911 24703 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu pid 18744
I20260809 23:04:43.472254 18811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 23:04:43.581419 24703 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHiIJT1/build/asan/bin/kudu with pid 18744
2026-08-09T23:04:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4549 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9183 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (372169 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST